Concurrent write operations for use with multi-threaded file logging

ABSTRACT

A data storage system for use with a multi-threaded processing system receives concurrent requests to store data to a common data store, and efficiently and securely swaps an active data store for a new data store while avoiding conflicts arising from multiple threads attempting to swap a same data store and minimizing reliance on operations that re-attempt actions upon failure of an attempted action, thereby improving performance of the data storage system and also the multi-threaded processing system.

BACKGROUND

Many computing systems execute multiple threads simultaneously, orconcurrently. Multiple threads or multiple processes may attempt towrite data simultaneously, or concurrently, to a common data store ormemory. When different threads attempt to write or otherwise log data toa common file or data store simultaneously, or otherwise concurrently,those threads may conflict or otherwise compete with each other, causingthe system to experience data loss, data corruption, and/or delays. Somesystems attempt to overcome such conflicts by providing a separate filewhere each thread can write data, which increases the amount of memoryand storage needed. Other systems attempt to overcome thread conflictsby locking threads and enabling only one thread to write to a data storeat any given point in time, but such thread-locking techniques sufferfrom a decrease in productivity and efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a computer network system, according to some embodiments.

FIG. 2 depicts a general computer system, according to some embodiments.

FIG. 3 depicts a block diagram for implementing concurrent storeoperations for multiple threads, according to some embodiments.

FIG. 4 depicts a high-level flowchart illustrating a method for storingdata in one or more data stores, according to some embodiments.

FIG. 5A-5I depict examples of data requests written to a data store,according to some embodiments.

FIG. 6 depicts an example implementation of a data storage system,according to some embodiments.

FIG. 7 depicts another example implementation of a data storage system,according to some embodiments.

DETAILED DESCRIPTION

The disclosed embodiments generally relate to methods and systems forallowing concurrent access, e.g. write/store operations, to a datastructure, e.g. a file, memory, or other data store, by multipleprocesses or threads and, in particular, for allowing concurrent, orotherwise substantially simultaneously, appending of data to a filethereby. A thread is a section of software code or program instructionsbeing processed. A software application includes at least one thread,but may include multiple threads that are processed concurrently, andmay all attempt to store data to a common data store. Eachrequestor/process/thread requests the amount of space it needs to appendor store its data. Upon receiving such requests, the disclosedembodiments attempt to allocate or reserve the requested amount of spaceand, when it is determined that a write request will exceed theavailable space in a data store, determine which requestingprocess/thread should be responsible for ensuring that all data iswritten to the active data store and designating/selecting a new datastore for subsequent write requests, thus avoiding conflicts betweenthreads that may each otherwise attempt to clean or “roll” the file asdiscussed herein. The disclosed embodiments accordingly efficientlyhandle creation/opening of new files, referred to as “file rolling,”when data that should be stored cannot be stored in the current file'sremaining available capacity. The disclosed embodiments may providemechanisms which ensure minimal delay and mitigate multi-thread conflictwhen opening or otherwise creating a new file, or new data store.

It should be appreciated that the disclosed embodiments may operate withfiles or other resources having an unlimited size, or otherwisecharacterized by a size large enough to hold all possible data thatcould be stored in a particular implementation, and thereby notrequiring another file or resource.

The disclosed embodiments may include a data storage system for periodiclogging of data regarding the operational state of a computer system,such as a match engine of an electronic trading system, whereby multiplethreads, processes or other programs, referred to as “loggers” or“logging threads”, monitor various portions of the operation of thecomputer system and record their observations, and/or other dataindicative thereof, in a log data file or other data structure. Itshould be appreciated that the more granular the monitoring, i.e. themore monitoring threads/process that can be implemented to monitor moreparts of the system and/or the rate or frequency at which those partsmay be monitored and data indicative thereof recorded, the more usefulthe overall monitoring function may be. Each thread/process may bemonitoring a different portion of the system and the computer system maybe operating at high speed, thereby generating a significant amount ofmonitored data in a short amount of time, all of which may be written bymultiple threads to a same file or data store.

While monitoring of system operation may be considered tangential to theoverall function of a system, the performance of the monitoringthreads/processes may nevertheless impact the performance of the systembecause those processes/threads typically use the same resources, e.g.processor/CPU, memory, etc., as the system being monitored. Therefore,it may be desirable that the monitoring threads/processes operateefficiently and with minimal impact on the operation of the system. Inparticular, as the operation of appending data to a data file may be arelatively time/resource consuming operation, the disclosed embodimentsimprove the performance of the monitoring threads/processes, and therebyimprove the performance of the system being monitored, by minimizing thecontention among the monitoring processes/threads for access to the datastore/file in which the monitored data is stored to append theirmonitored data to an active data store, and by minimizing uncertaintyabout which process/thread should roll the file so that subsequentrequests are stored in a new data store.

Using the disclosed embodiments, each process/thread can determine if itis or is not the one thread that should roll the file and select a newfile for subsequent data write operations, and, based on thesedeterminations, only that one thread will roll the file, thuseliminating file rolling conflicts which can corrupt the data orseverely slow down the system.

In one embodiment, the system may comprise a match engine of anelectronic trading system and the monitoring threads/processes maycontinuously monitor and record time stamp and address data indicativeof the timing of particular operations/functions/milestones/states ofthe system for the purpose of system performance evaluation,problem/defect resolutions, historical data analysis, operationalscenario reconstruction, regulatory or administrative audit, or otherpurpose. However, it will be appreciated that disclosed embodiments maybe utilized in conjunction with any system or multi-threadedimplementation where multiple processes/threads need to append data to ashared resource, e.g. file or other data store.

As used herein, the terms concurrent and/or substantially simultaneouslyrefer to the ability of one thread process to commence, but notnecessarily complete, its operation to append data to a file or otherresource before another process/thread completes a previously commencedoperation to append data to the same file or resource, or otherwisewhere the operations may overlap in time.

The disclosed embodiments may be implemented separately from, or as partof, the hardware and/or an operating system upon which the monitoredsystem is implemented, as part of the monitored system, as part of themonitoring/logging system, or combinations thereof and all suchimplementations are contemplated. In one embodiment, the disclosedfunctionality may be implemented as part of a file management system orother supervisory process to which other threads/processes request fileaccess. The disclosed embodiments may be implemented, as describedbelow, as computer program code stored in a memory or other volatile ornon-volatile data storage device and/or as a hardware component such asa programmable or reprogrammable logic device or an application specificintegrated circuit. The disclosed embodiments may be implemented as adata storage system that stores/records data about a monitored system.

The disclosed embodiments particularly relate to allowing multiplethreads/processes to concurrently append new data to a file, i.e. addnew data to, and thereby, increase the size of the file. Using thedisclosed embodiments, the output of multiple threads may be captured ina single file or set of files with minimal impact on the performance ofthe system being monitored as was described. Where the output of thethreads may be interrelated and characterized by a order, e.g. sequenceor time order, in which the output was generated by each thread,capturing to a single file, or set of files, may require less postprocessing of the collected data.

For example, an alternative solution to the resource contention issuedescribed herein may be to provide a separate resource/file for eachthread to write to. However, where the data is interrelated such via anorder, such as a time order or sequence, these separate files may needto be post processed to combine the data for subsequent analysis orprocessing. This may not be a trivial task. Further, the maintenance ofmultiple open files by the operating system on which the monitoring isoccurring, may itself cause performance degradation.

Another solution to the resource contention issue described herein maybe to only enable one thread to write, at any one time, to a commonresource/file. However, such locking of all threads except for thewriting thread may significantly degrade system performance.

In one embodiment, systems and methods are disclosed for storing datarelated to monitoring operations of an electronic trading system using amonitoring system having multiple logging processes/threads monitoringvarious portions of the electronic trading system and concurrentlystoring the monitored data in a common log file or other data store inaccordance therewith.

The disclosed embodiments may determine which of many requestors shouldbe responsible for cleaning a data store and rollingadditional/subsequent data write operations to a new data store. Thedisclosed embodiments advantageously allow multiple threads tosimultaneously write data to a common data store, and enable selectionof a new data store when an active data store is full or almost full,without the multiple threads conflicting with each other while writingdata or while rolling the file.

In one embodiment, an individual file's state may be represented byindependent counters, e.g., an allocation counter, indicative of whichportions of a data store have been allocated for data storage, and anacknowledgement counter, indicative of portions of a data store wheredata has actually been written in the data store.

A counter may be implemented in hardware, e.g., a counting device, ormay be implemented in software and may that count and be characterizedby current values, increments, etc. For example, a counter may incrementor decrement a value by an increment/decrement amount. The value may bean address/pointer that points to a position in the data store. Or thevalue may be an offset, added to a base value (which in some cases couldbe zero), to calculate a position in the data store.

In one embodiment, a value of an allocation counter may be returned to arequestor and incremented atomically. In another embodiment, the valueof an allocation counter may be returned to a requestor after the valueof the allocation counter is incremented. It should be understood thatan atomic operation or multiple operations that operate togetheratomically are operations that are either all performed or not performedat all. For example, if two steps are performed as an atomic operation,each of the two steps is committed to memory and is allowed to changethe state of the system only if the other step is also committed tomemory and is allowed to change the state of the system. An operation(or set of operations) is atomic if it appears to the rest of the systemto occur instantaneously. Atomicity is a guarantee of isolation frominterrupts, signals, concurrent processes and threads. An operation thatis atomic may also be referred to as linearizable, indivisible oruninterruptible. A thread acting on shared memory is atomic if itcompletes in a single step relative to other threads. For example, whenan atomic store is performed on a shared variable, no other thread canobserve the modification half-complete. When an atomic load is performedon a shared variable, it reads the entire value as it appeared at asingle moment in time. Non-atomic loads and stores do not provide suchguarantees.

The disclosed embodiments also reduce the reliance on compare and swap(“CAS”) type operations, or similar operations, which are operationsthat attempt to perform an action, where the attempt may be successfulor may fail, and where if the attempt to perform an action fails, theoperation re-attempts to perform the same action, until the action canbe successfully performed. Implementing CAS-type operations may beinefficient because CAS-type operations are often implemented wherethere is a high probability (e.g., close to 50% in some cases) that theattempted action will fail, causing the CAS-type operation to attempt toperform the action again.

U.S. Patent Publication No. 2016/0328435 entitled “Thread Safe Lock-FreeConcurrent Write Operations For Use With Multi-Threaded In-Line Logging”and filed on May 8, 2015 (“the '435 Publication”), assigned to theassignee of the present application, the entire disclosure of which isincorporated by reference herein and relied upon, describes embodimentsfor allowing multiple threads to concurrently append new data to a filethat implement CAS operations.

Due to very nature of CAS-type operations, i.e., because they areimplemented where an attempted action is expected to fail during atleast some of the attempts, they increase the amount of software loopsexecuted by the processes, and may be inefficient. In one embodiment,the disclosed embodiments minimize the reliance on potentially repeatingsoftware loops and improve upon the speed and efficiency with whichmultiple threads can write to a common file or data store. By reducingthe amount of potentially repeating software loops, the disclosedembodiments enable faster and more reliable data write operations.

In one embodiment, the data storage system does not implement CAS-typeoperations to allocate data, write data to the data store, or to swapold/used files. In another embodiment, the data storage systemoptionally implements one CAS-type operation to swap old/used files, butstill does not implement CAS-type operations to allocate data, or towrite data to the data store. The disclosed embodiments accordinglyimprove upon technologies that extensively implement CAS-typeoperations. The disclosed embodiments accordingly improve upon thetechnical field of file logging and/or data storage in a multi-threadedsystem, such as a multi-threaded monitoring system. At least some of theproblems solved by the disclosed embodiments are specifically rooted intechnology, where different threads/processes attempt to write to ashared data store, and where the data store may be of a fixed orvariable size that is periodically closed and replaced by anew/different data store.

The disclosed embodiments described herein also improve the technologyof data processing and computer implemented system performance and, inparticular, the data storage system associated with logging ofinformation, e.g., of a monitored system. By improving the mechanism forconcurrent storage of data to a common data store, the disclosedembodiments eliminate the need for file locks or othersynchronization/serialization techniques and/or reduce the reliance onoperations that re-attempt actions upon failure of an attempted action,thereby improving performance of the data storage system and, thereby,the system about which data is recorded. Furthermore, the disclosedembodiments minimize delay incurred due to swapping the common datastore, when it reaches its capacity, for a new data store which furtherimproves the performance of the data storage system and, thereby, thesystem about which data is recorded. The disclosed embodiments providethe described performance improvements while allowing for the creationof a single log files containing the data from the multiple monitoringthreads/processes. This alleviates the need, and the necessaryresources, to post-process numerous log files in order to combine thethread/process output for further analysis. It will be appreciated thatby improving the performance of the data storage system, the performanceof the system about which data is recorded may be improved and/oradditional and/or more granular monitoring may be performed withoutimpacting the performance of the monitored system.

The disclosed embodiments improve data storage systems that allow formulti-thread logging in several ways. The disclosed embodimentsfacilitate ensuring that no two threads write to the samelocation/addresses in the file at the same time, that a file is notcleaned out of memory (flushed to disk and closed) by one thread whileanother thread attempts to write to the file, and that a file is notcleaned multiple times (e.g., by multiple different threads). Thedisclosed embodiments also allow a system to specify a maximum filesize, such that the data store size does not exceed the specifiedmaximum file size.

Exchange Computing System

The disclosed embodiments may be implemented in a data transactionprocessing system that processes data items or objects. Customer or userdevices (e.g., client computers) may submit electronic data transactionrequest messages, e.g., inbound messages, to the data transactionprocessing system over a data communication network. The electronic datatransaction request messages may include, for example, transactionmatching parameters, such as instructions and/or values, for processingthe data transaction request messages within the data transactionprocessing system. The instructions may be to perform transactions,e.g., buy or sell a quantity of a product at a range of values definedequations. Products, e.g., financial instruments, or order booksrepresenting the state of an electronic marketplace for a product, maybe represented as data objects within the exchange computing system. Theinstructions may also be conditional, e.g., buy or sell a quantity of aproduct at a given value if a trade for the product is executed at someother reference value. The data transaction processing system mayinclude various specifically configured matching processors that match,e.g., automatically, electronic data transaction request messages forthe same one of the data items or objects. The specifically configuredmatching processors may match, or attempt to match, electronic datatransaction request messages based on multiple transaction matchingparameters from the different client computers. The specificallyconfigured matching processors may additionally generate informationindicative of a state of an environment (e.g., the state of the orderbook) based on the processing, and report this information to datarecipient computing systems via outbound messages published via one ormore data feeds.

For example, one exemplary environment where the disclosed embodimentsmay be desirable is in financial markets, and in particular, electronicfinancial exchanges, such as a futures exchange, such as the ChicagoMercantile Exchange Inc. (CME).

A financial instrument trading system, such as a futures exchange, suchas the Chicago Mercantile Exchange Inc. (CME), provides a contractmarket where financial instruments, e.g., futures and options onfutures, are traded using electronic systems. “Futures” is a term usedto designate all contracts for the purchase or sale of financialinstruments or physical commodities for future delivery or cashsettlement on a commodity futures exchange. A futures contract is alegally binding agreement to buy or sell a commodity at a specifiedprice at a predetermined future time. An option contract is the right,but not the obligation, to sell or buy the underlying instrument (inthis case, a futures contract) at a specified price on or before acertain expiration date. An option contract offers an opportunity totake advantage of futures price moves without actually having a futuresposition. The commodity to be delivered in fulfillment of the contract,or alternatively the commodity for which the cash market price shalldetermine the final settlement price of the futures contract, is knownas the contract's underlying reference or “underlier.” The underlying orunderlier for an options contract is the corresponding futures contractthat is purchased or sold upon the exercise of the option.

The terms and conditions of each futures contract are standardized as tothe specification of the contract's underlying reference commodity, thequality of such commodity, quantity, delivery date, and means ofcontract settlement. Cash settlement is a method of settling a futurescontract whereby the parties effect final settlement when the contractexpires by paying/receiving the loss/gain related to the contract incash, rather than by effecting physical sale and purchase of theunderlying reference commodity at a price determined by the futurescontract, price. Options and futures may be based on more generalizedmarket indicators, such as stock indices, interest rates, futurescontracts and other derivatives.

An exchange may provide for a centralized “clearing house” through whichtrades made must be confirmed, matched, and settled each day untiloffset or delivered. The clearing house may be an adjunct to anexchange, and may be an operating division of an exchange, which isresponsible for settling trading accounts, clearing trades, collectingand maintaining performance bond funds, regulating delivery, andreporting trading data. One of the roles of the clearing house is tomitigate credit risk. Clearing is the procedure through which theclearing house becomes buyer to each seller of a futures contract, andseller to each buyer, also referred to as a novation, and assumesresponsibility for protecting buyers and sellers from financial loss dueto breach of contract, by assuring performance on each contract. Aclearing member is a firm qualified to clear trades through the clearinghouse.

An exchange computing system may operate under a central counterpartymodel, where the exchange acts as an intermediary between marketparticipants for the transaction of financial instruments. Inparticular, the exchange computing system novates itself into thetransactions between the market participants, i.e., splits a giventransaction between the parties into two separate transactions where theexchange computing system substitutes itself as the counterparty to eachof the parties for that part of the transaction, sometimes referred toas a novation. In this way, the exchange computing system acts as aguarantor and central counterparty and there is no need for the marketparticipants to disclose their identities to each other, or subjectthemselves to credit or other investigations by a potentialcounterparty. For example, the exchange computing system insulates onemarket participant from the default by another market participant.Market participants need only meet the requirements of the exchangecomputing system. Anonymity among the market participants encourages amore liquid market environment as there are lower barriers toparticipation. The exchange computing system can accordingly offerbenefits such as centralized and anonymous matching and clearing.

Clearing House

The clearing house of an exchange clears, settles and guarantees matchedtransactions in contracts occurring through the facilities of theexchange. In addition, the clearing house establishes and monitorsfinancial requirements for clearing members and conveys certain clearingprivileges in conjunction with the relevant exchange markets.

The clearing house establishes clearing level performance bonds(margins) for all products of the exchange and establishes minimumperformance bond requirements for customers of such products. Aperformance bond, also referred to as a margin requirement, correspondswith the funds that must be deposited by a customer with his or herbroker, by a broker with a clearing member or by a clearing member withthe clearing house, for the purpose of insuring the broker or clearinghouse against loss on open futures or options contracts. This is not apart payment on a purchase. The performance bond helps to ensure thefinancial integrity of brokers, clearing members and the exchange as awhole. The performance bond refers to the minimum dollar depositrequired by the clearing house from clearing members in accordance withtheir positions. Maintenance, or maintenance margin, refers to a sum,usually smaller than the initial performance bond, which must remain ondeposit in the customer's account for any position at all times. Theinitial margin is the total amount of margin per contract required bythe broker when a futures position is opened. A drop in funds below thislevel requires a deposit back to the initial margin levels, i.e., aperformance bond call. If a customer's equity in any futures positiondrops to or under the maintenance level because of adverse price action,the broker must issue a performance bond/margin call to restore thecustomer's equity. A performance bond call, also referred to as a margincall, is a demand for additional funds to bring the customer's accountback up to the initial performance bond level whenever adverse pricemovements cause the account to go below the maintenance.

Electronic Data Transaction Request Messages

As used herein, a financial message, or an electronic message, refersboth to messages communicated by market participants to an electronictrading or market system and vice versa. The messages may becommunicated using packeting or other techniques operable to communicateinformation between systems and system components. Some messages may beassociated with actions to be taken in the electronic trading or marketsystem. Financial messages communicated to the electronic tradingsystem, also referred to as “inbound” messages, may include associatedactions that characterize the messages, such as trader orders, ordermodifications, order cancellations and the like, as well as othermessage types. Inbound messages may be sent from market participants, ortheir representatives, e.g., trade order messages, etc., to anelectronic trading or market system. For example, a market participantmay submit an electronic message to the electronic trading system thatincludes an associated specific action to be undertaken by theelectronic trading system, such as entering a new trade order into themarket or modifying an existing order in the market. In one exemplaryembodiment, the incoming request itself, e.g., the inbound order entry,may be referred to as an iLink message. iLink is a bidirectionalcommunications/message protocol/message format implemented by theChicago Mercantile Exchange Inc.

Financial messages communicated from the electronic trading system,referred to as “outbound” messages, may include messages responsive toinbound messages, such as confirmation messages, or other messages suchas market update messages, quote messages, and the like. Outboundmessages may be disseminated via data feeds.

Financial messages may further be categorized as having or reflecting animpact on a market or electronic marketplace, also referred to as an“order book” or “book,” for a traded product, such as a prevailing pricetherefore, number of resting orders at various price levels andquantities thereof, etc., or not having or reflecting an impact on amarket or a subset or portion thereof. In one embodiment, an electronicorder book may be understood to be an electronic collection of theoutstanding or resting orders for a financial instrument.

For example, a request to place a trade may result in a responseindicative of the trade either being matched with, or being rested on anorder book to await, a suitable counter-order. This response may includea message directed solely to the trader who submitted the order toacknowledge receipt of the order and report whether it was matched, andthe extent thereto, or rested. The response may further include amessage to all market participants reporting a change in the order bookdue to the order. This response may take the form of a report of thespecific change to the order book, e.g., an order for quantity X atprice Y was added to the book (referred to, in one embodiment, as aMarket By Order message), or may simply report the result, e.g., pricelevel Y now has orders for a total quantity of Z (where Z is the sum ofthe previous resting quantity plus quantity X of the new order). In somecases, requests may elicit a non-impacting response, such as temporallyproximate to the receipt of the request, and then cause a separatemarket-impact reflecting response at a later time. For example, a stoporder, fill or kill order (FOK), also known as an immediate or cancelorder, or other conditional request may not have an immediate marketimpacting effect, if at all, until the requisite conditions are met.

An acknowledgement or confirmation of receipt, e.g., a non-marketimpacting communication, may be sent to the trader simply confirmingthat the order was received. Upon the conditions being met and a marketimpacting result thereof occurring, a market-impacting message may betransmitted as described herein both directly back to the submittingmarket participant and to all market participants (in a Market By Price“MBP” e.g., Aggregated By Value (“ABV”) book, or Market By Order “MBO”,e.g., Per Order (“PO”) book format). It should be appreciated thatadditional conditions may be specified, such as a time or price limit,which may cause the order to be dropped or otherwise canceled and thatsuch an event may result in another non-market-impacting communicationinstead. In some implementations, market impacting communications may becommunicated separately from non-market impacting communications, suchas via a separate communications channel or feed.

It should be further appreciated that various types of market data feedsmay be provided which reflect different markets or aspects thereof.Market participants may then, for example, subscribe to receive thosefeeds of interest to them. For example, data recipient computing systemsmay choose to receive one or more different feeds. As market impactingcommunications usually tend to be more important to market participantsthan non-impacting communications, this separation may reduce congestionand/or noise among those communications having or reflecting an impacton a market or portion thereof. Furthermore, a particular market datafeed may only communicate information related to the top buy/sell pricesfor a particular product, referred to as “top of book” feed, e.g., onlychanges to the top 10 price levels are communicated. Such limitationsmay be implemented to reduce consumption of bandwidth and messagegeneration resources. In this case, while a request message may beconsidered market-impacting if it affects a price level other than thetop buy/sell prices, it will not result in a message being sent to themarket participants.

Examples of the various types of market data feeds which may be providedby electronic trading systems, such as the CME, in order to providedifferent types or subsets of market information or to provide suchinformation in different formats include Market By Order or Per Order,Market Depth (also known as Market by Price or Aggregated By Value to adesignated depth of the book), e.g., CME offers a 10-deep market byprice feed, Top of Book (a single depth Market by Price feed), andcombinations thereof. There may also be all manner of specialized feedsin terms of the content, i.e., providing, for example, derived data,such as a calculated index.

Market data feeds may be characterized as providing a “view” or“overview” of a given market, an aggregation or a portion thereof orchanges thereto. For example, a market data feed, such as a Market ByPrice (“MBP”) feed, also known as an Aggregated By Value (“ABV”) feed,may convey, with each message, the entire/current state of a market, orportion thereof, for a particular product as a result of one or moremarket impacting events. For example, an MBP message may convey a totalquantity of resting buy/sell orders at a particular price level inresponse to a new order being placed at that price. An MBP message mayconvey a quantity of an instrument which was traded in response to anincoming order being matched with one or more resting orders. MBPmessages may only be generated for events affecting a portion of amarket, e.g., only the top 10 resting buy/sell orders and, thereby, onlyprovide a view of that portion. As used herein, a market impactingrequest may be said to impact the “view” of the market as presented viathe market data feed.

An MBP feed may utilize different message formats for conveyingdifferent types of market impacting events. For example, when a neworder is rested on the order book, an MBP message may reflect thecurrent state of the price level to which the order was added, e.g., thenew aggregate quantity and the new aggregate number of resting orders.As can be seen, such a message conveys no information about theindividual resting orders, including the newly rested order, themselvesto the market participants. Only the submitting market participant, whoreceives a separate private message acknowledging the event, knows thatit was their order that was added to the book. Similarly, when a tradeoccurs, an MBP message may be sent which conveys the price at which theinstrument was traded, the quantity traded and the number ofparticipating orders, but may convey no information as to whoseparticular orders contributed to the trade. MBP feeds may further batchreporting of multiple events, i.e., report the result of multiple marketimpacting events in a single message.

Alternatively, a market data feed, referred to as a Market By Order(“MBO”) feed also known as a Per Order (“PO”) feed, may convey datareflecting a change that occurred to the order book rather than theresult of that change, e.g., that order ABC for quantity X was added toprice level Y or that order ABC and order XYZ traded a quantity X at aprice Y. In this case, the MBO message identifies only the change thatoccurred so a market participant wishing to know the current state ofthe order book must maintain their own copy and apply the changereflected in the message to know the current state. As can be seen,MBO/PO messages may carry much more data than MBP/ABV messages becauseMBO/PO messages reflect information about each order, whereas MBP/ABVmessages contain information about orders affecting some predeterminedvalue levels. Furthermore, because specific orders, but not thesubmitting traders thereof, are identified, other market participantsmay be able to follow that order as it progresses through the market,e.g., as it is modified, canceled, traded, etc.

An ABV book data object may include information about multiple values.The ABV book data object may be arranged and structured so thatinformation about each value is aggregated together. Thus, for a givenvalue V, the ABV book data object may aggregate all the information byvalue, such as for example, the number of orders having a certainposition at value V, the quantity of total orders resting at value V,etc. Thus, the value field may be the key, or may be a unique field,within an ABV book data object. In one embodiment, the value for eachentry within the ABV book data object is different. In one embodiment,information in an ABV book data object is presented in a manner suchthat the value field is the most granular field of information.

A PO book data object may include information about multiple orders. ThePO book data object may be arranged and structured so that informationabout each order is represented. Thus, for a given order O, the PO bookdata object may provide all of the information for order O. Thus, theorder field may be the key, or may be a unique field, within a PO bookdata object. In one embodiment, the order ID for each entry within thePO book data object is different. In one embodiment, information in a PObook data object is presented in a manner such that the order field isthe most granular field of information.

Thus, the PO book data object may include data about unique orders,e.g., all unique resting orders for a product, and the ABV book dataobject may include data about unique values, e.g., up to a predeterminedlevel, e.g., top ten price or value levels, for a product.

It should be appreciated that the number, type and manner of market datafeeds provided by an electronic trading system are implementationdependent and may vary depending upon the types of products traded bythe electronic trading system, customer/trader preferences, bandwidthand data processing limitations, etc. and that all such feeds, nowavailable or later developed, are contemplated herein. MBP/ABV andMBO/PO feeds may refer to categories/variations of market data feeds,distinguished by whether they provide an indication of the current stateof a market resulting from a market impacting event (MBP) or anindication of the change in the current state of a market due to amarket impacting event (MBO).

Messages, whether MBO or MBP, generated responsive to market impactingevents which are caused by a single order, such as a new order, an ordercancellation, an order modification, etc., are fairly simple and compactand easily created and transmitted. However, messages, whether MBO orMBP, generated responsive to market impacting events which are caused bymore than one order, such as a trade, may require the transmission of asignificant amount of data to convey the requisite information to themarket participants. For trades involving a large number of orders,e.g., a buy order for a quantity of 5000 which matches 5000 sell orderseach for a quantity of 1, a significant amount of information may needto be sent, e.g., data indicative of each of the 5000 trades that haveparticipated in the market impacting event.

In one embodiment, an exchange computing system may generate multipleorder book objects, one for each type of view that is published orprovided. For example, the system may generate a PO book object and anABV book object. It should be appreciated that each book object, or viewfor a product or market, may be derived from the Per Order book object,which includes all the orders for a given financial product or market.

An inbound message may include an order that affects the PO book object,the ABV book object, or both. An outbound message may include data fromone or more of the structures within the exchange computing system,e.g., the PO book object queues or the ABV book object queues.

Furthermore, each participating trader needs to receive a notificationthat their particular order has traded. Continuing with the example,this may require sending 5001 individual trade notification messages, oreven 10,000+ messages where each contributing side (buy vs. sell) isseparately reported, in addition to the notification sent to all of themarket participants.

Market Segment Gateway

In one embodiment, the disclosed system may include a Market SegmentGateway (“MSG”) that is the point of ingress/entry and/oregress/departure for all transactions, i.e., the network traffic/packetscontaining the data therefore, specific to a single market at which theorder of receipt of those transactions may be ascribed. An MSG or MarketSegment Gateway may be utilized for the purpose of deterministicoperation of the market. The electronic trading system may includemultiple markets, and because the electronic trading system includes oneMSG for each market/product implemented thereby, the electronic tradingsystem may include multiple MSGs. For more detail on deterministicoperation in a trading system, see U.S. Patent Publication No.2015/0127513 entitled “Transactionally Deterministic High SpeedFinancial Exchange Having Improved, Efficiency, Communication,Customization, Performance, Access, Trading Opportunities, CreditControls, And Fault Tolerance” and filed on Nov. 7, 2013 (“the '513Publication”), the entire disclosure of which is incorporated byreference herein and relied upon.

For example, a participant may send a request for a new transaction,e.g., a request for a new order, to the MSG. The MSG extracts or decodesthe request message and determines the characteristics of the requestmessage.

The MSG may include, or otherwise be coupled with, a buffer, cache,memory, database, content addressable memory, data store or other datastorage mechanism, or combinations thereof, which stores data indicativeof the characteristics of the request message. The request is passed tothe transaction processing system, e.g., the match engine.

An MSG or Market Segment Gateway may be utilized for the purpose ofdeterministic operation of the market. Transactions for a particularmarket may be ultimately received at the electronic trading system viaone or more points of entry, e.g., one or more communicationsinterfaces, at which the disclosed embodiments apply determinism, whichas described may be at the point where matching occurs, e.g., at eachmatch engine (where there may be multiple match engines, each for agiven product/market, or moved away from the point where matching occursand closer to the point where the electronic trading system firstbecomes “aware” of the incoming transaction, such as the point wheretransaction messages, e.g., orders, ingress the electronic tradingsystem. Generally, the terms “determinism” or “transactionaldeterminism” may refer to the processing, or the appearance thereof, oforders in accordance with defined business rules. Accordingly, as usedherein, the point of determinism may be the point at which theelectronic trading system ascribes an ordering to incomingtransactions/orders relative to other incoming transactions/orders suchthat the ordering may be factored into the subsequent processing, e.g.,matching, of those transactions/orders as will be described. For moredetail on deterministic operation in a trading system, see the '513Publication.

Electronic Trading

Electronic trading of financial instruments, such as futures contracts,is conducted by market participants sending orders, such as to buy orsell one or more futures contracts, in electronic form to the exchange.These electronically submitted orders to buy and sell are then matched,if possible, by the exchange, i.e., by the exchange's matching engine,to execute a trade. Outstanding (unmatched, wholly unsatisfied/unfilledor partially satisfied/filled) orders are maintained in one or more datastructures or databases referred to as “order books,” such orders beingreferred to as “resting,” and made visible, i.e., their availability fortrading is advertised, to the market participants through electronicnotifications/broadcasts, referred to as market data feeds. An orderbook is typically maintained for each product, e.g., instrument, tradedon the electronic trading system and generally defines or otherwiserepresents the state of the market for that product, i.e., the currentprices at which the market participants are willing buy or sell thatproduct. As such, as used herein, an order book for a product may alsobe referred to as a market for that product.

Upon receipt of an incoming order to trade in a particular financialinstrument, whether for a single-component financial instrument, e.g., asingle futures contract, or for a multiple-component financialinstrument, e.g., a combination contract such as a spread contract, amatch engine, as described herein, will attempt to identify a previouslyreceived but unsatisfied order counter thereto, i.e., for the oppositetransaction (buy or sell) in the same financial instrument at the sameor better price (but not necessarily for the same quantity unless, forexample, either order specifies a condition that it must be entirelyfilled or not at all).

Previously received but unsatisfied orders, i.e., orders which eitherdid not match with a counter order when they were received or theirquantity was only partially satisfied, referred to as a partial fill,are maintained by the electronic trading system in an order bookdatabase/data structure to await the subsequent arrival of matchingorders or the occurrence of other conditions which may cause the orderto be modified or otherwise removed from the order book.

If the match engine identifies one or more suitable previously receivedbut unsatisfied counter orders, they, and the incoming order, arematched to execute a trade there between to at least partially satisfythe quantities of one or both the incoming order or the identifiedorders. If there remains any residual unsatisfied quantity of theidentified one or more orders, those orders are left on the order bookwith their remaining quantity to await a subsequent suitable counterorder, i.e., to rest. If the match engine does not identify a suitablepreviously received but unsatisfied counter order, or the one or moreidentified suitable previously received but unsatisfied counter ordersare for a lesser quantity than the incoming order, the incoming order isplaced on the order book, referred to as “resting”, with original orremaining unsatisfied quantity, to await a subsequently receivedsuitable order counter thereto. The match engine then generates matchevent data reflecting the result of this matching process. Othercomponents of the electronic trading system, as will be described, thengenerate the respective order acknowledgment and market data messagesand transmit those messages to the market participants.

Matching, which is a function typically performed by the exchange, is aprocess, for a given order which specifies a desire to buy or sell aquantity of a particular instrument at a particular price, ofseeking/identifying one or more wholly or partially, with respect toquantity, satisfying counter orders thereto, e.g., a sell counter to anorder to buy, or vice versa, for the same instrument at the same, orsometimes better, price (but not necessarily the same quantity), whichare then paired for execution to complete a trade between the respectivemarket participants (via the exchange) and at least partially satisfythe desired quantity of one or both of the order and/or the counterorder, with any residual unsatisfied quantity left to await anothersuitable counter order, referred to as “resting.” A match event mayoccur, for example, when an aggressing order matches with a restingorder. In one embodiment, two orders match because one order includesinstructions for or specifies buying a quantity of a particularinstrument at a particular price, and the other order includesinstructions for or specifies selling a (different or same) quantity ofthe instrument at a same or better price. It should be appreciated thatperforming an instruction associated with a message may includeattempting to perform the instruction. Whether or not an exchangecomputing system is able to successfully perform an instruction maydepend on the state of the electronic marketplace.

While the disclosed embodiments will be described with respect to aproduct by product or market by market implementation, e.g. implementedfor each market/order book, it will be appreciated that the disclosedembodiments may be implemented so as to apply across markets formultiple products traded on one or more electronic trading systems, suchas by monitoring an aggregate, correlated or other derivation of therelevant indicative parameters as described herein.

While the disclosed embodiments may be discussed in relation to futuresand/or options on futures trading, it should be appreciated that thedisclosed embodiments may be applicable to any equity, fixed incomesecurity, currency, commodity, options or futures trading system ormarket now available or later developed. It may be appreciated that atrading environment, such as a futures exchange as described herein,implements one or more economic markets where rights and obligations maybe traded. As such, a trading environment may be characterized by a needto maintain market integrity, transparency, predictability,fair/equitable access and participant expectations with respect thereto.In addition, it may be appreciated that electronic trading systemsfurther impose additional expectations and demands by marketparticipants as to transaction processing speed, latency, capacity andresponse time, while creating additional complexities relating thereto.Accordingly, as will be described, the disclosed embodiments may furtherinclude functionality to ensure that the expectations of marketparticipants are met, e.g., that transactional integrity and predictablesystem responses are maintained.

Financial instrument trading systems allow traders to submit orders andreceive confirmations, market data, and other information electronicallyvia electronic messages exchanged using a network. Electronic tradingsystems ideally attempt to offer a more efficient, fair and balancedmarket where market prices reflect a true consensus of the value oftraded products among the market participants, where the intentional orunintentional influence of any one market participant is minimized ifnot eliminated, and where unfair or inequitable advantages with respectto information access are minimized if not eliminated.

Electronic marketplaces attempt to achieve these goals by usingelectronic messages to communicate actions and related data of theelectronic marketplace between market participants, clearing firms,clearing houses, and other parties. The messages can be received usingan electronic trading system, wherein an action or transactionassociated with the messages may be executed. For example, the messagemay contain information relating to an order to buy or sell a product ina particular electronic marketplace, and the action associated with themessage may indicate that the order is to be placed in the electronicmarketplace such that other orders which were previously placed maypotentially be matched to the order of the received message. Thus theelectronic marketplace may conduct market activities through electronicsystems.

The exchange derives its financial stability in large part by removingdebt obligations among market participants as they occur. This isaccomplished by determining a settlement price at the close of themarket each day for each contract and marking all open positions to thatprice, referred to as “mark to market.” Every contract is debited orcredited based on that trading session's gains or losses. As prices movefor or against a position, funds flow into and out of the tradingaccount. In the case of the CME, each business day by 6:40 a.m. Chicagotime, based on the mark-to-the-market of all open positions to theprevious trading day's settlement price, the clearing house pays to orcollects cash from each clearing member. This cash flow, known assettlement variation, is performed by CME's settlement banks based oninstructions issued by the clearing house. All payments to andcollections from clearing members are made in “same-day” funds. Inaddition to the 6:40 a.m. settlement, a daily intra-day mark-to-themarket of all open positions, including trades executed during theovernight GLOBEX®, the CME's electronic trading systems, trading sessionand the current day's trades matched before 11:15 a.m., is performedusing current prices. The resulting cash payments are made intra-day forsame day value. In times of extreme price volatility, the clearing househas the authority to perform additional intra-day mark-to-the-marketcalculations on open positions and to call for immediate payment ofsettlement variation. CME's mark-to-the-market settlement system differsfrom the settlement systems implemented by many other financial markets,including the interbank, Treasury securities, over-the-counter foreignexchange and debt, options, and equities markets, where participantsregularly assume credit exposure to each other. In those markets, thefailure of one participant can have a ripple effect on the solvency ofthe other participants. Conversely, CME's mark-to-the-market system doesnot allow losses to accumulate over time or allow a market participantthe opportunity to defer losses associated with market positions.

While the disclosed embodiments may be described in reference to theCME, it should be appreciated that these embodiments are applicable toany exchange. Such other exchanges may include a clearing house that,like the CME clearing house, clears, settles and guarantees all matchedtransactions in contracts of the exchange occurring through itsfacilities. In addition, such clearing houses establish and monitorfinancial requirements for clearing members and convey certain clearingprivileges in conjunction with the relevant exchange markets.

The disclosed embodiments are also not limited to uses by a clearinghouse or exchange for purposes of enforcing a performance bond or marginrequirement. For example, a market participant may use the disclosedembodiments in a simulation or other analysis of a portfolio. In suchcases, the settlement price may be useful as an indication of a value atrisk and/or cash flow obligation rather than a performance bond. Thedisclosed embodiments may also be used by market participants or otherentities to forecast or predict the effects of a prospective position onthe margin requirement of the market participant.

Trading Environment

The embodiments may be described in terms of a distributed computingsystem. The particular examples identify a specific set of componentsuseful in a futures and options exchange. However, many of thecomponents and inventive features are readily adapted to otherelectronic trading environments. The specific examples described hereinmay teach specific protocols and/or interfaces, although it should beunderstood that the principles involved may be extended to, or appliedin, other protocols and interfaces.

It should be appreciated that the plurality of entities utilizing orinvolved with the disclosed embodiments, e.g., the market participants,may be referred to by other nomenclature reflecting the role that theparticular entity is performing with respect to the disclosedembodiments and that a given entity may perform more than one roledepending upon the implementation and the nature of the particulartransaction being undertaken, as well as the entity's contractual and/orlegal relationship with another market participant and/or the exchange.

An exemplary trading network environment for implementing tradingsystems and methods is shown in FIG. 1. An exchange computer system 100receives messages that include orders and transmits market data relatedto orders and trades to users, such as via wide area network 162 and/orlocal area network 160 and computer devices 150, 152, 154, 156 and 158,as described herein, coupled with the exchange computer system 100.

Herein, the phrase “coupled with” is defined to mean directly connectedto or indirectly connected through one or more intermediate components.Such intermediate components may include both hardware and softwarebased components. Further, to clarify the use in the pending claims andto hereby provide notice to the public, the phrases “at least one of<A>, <B>, . . . and <N>” or “at least one of <A>, <B>, . . . <N>, orcombinations thereof” are defined by the Applicant in the broadestsense, superseding any other implied definitions herebefore orhereinafter unless expressly asserted by the Applicant to the contrary,to mean one or more elements selected from the group comprising A, B, .. . and N, that is to say, any combination of one or more of theelements A, B, . . . or N including any one element alone or incombination with one or more of the other elements which may alsoinclude, in combination, additional elements not listed.

The exchange computer system 100 may be implemented with one or moremainframe, desktop or other computers, such as the example computer 200described herein with respect to FIG. 2. A user database 102 may beprovided which includes information identifying traders and other usersof exchange computer system 100, such as account numbers or identifiers,user names and passwords. An account data module 104 may be providedwhich may process account information that may be used during trades.

A match engine module 106 may be included to match bid and offer pricesand may be implemented with software that executes one or morealgorithms for matching bids and offers. A trade database 108 may beincluded to store information identifying trades and descriptions oftrades. In particular, a trade database may store informationidentifying the time that a trade took place and the contract price. Anorder book module 110 may be included to compute or otherwise determinecurrent bid and offer prices, e.g., in a continuous auction market, oralso operate as an order accumulation buffer for a batch auction market.

A market data module 112 may be included to collect market data andprepare the data for transmission to users.

A risk management module 114 may be included to compute and determine auser's risk utilization in relation to the user's defined riskthresholds. The risk management module 114 may also be configured todetermine risk assessments or exposure levels in connection withpositions held by a market participant. The risk management module 114may be configured to administer, manage or maintain one or moremargining mechanisms implemented by the exchange computer system 100.Such administration, management or maintenance may include managing anumber of database records reflective of margin accounts of the marketparticipants. In some embodiments, the risk management module 114implements one or more aspects of the disclosed embodiments, including,for instance, principal component analysis (PCA) based margining, inconnection with interest rate swap (IRS) portfolios, as describedherein.

A message management module 116 may be included to, among other things,receive, and extract orders from, electronic data transaction requestmessages. The message management module 116 may define a point ofingress into the exchange computer system 100 where messages are orderedand considered to be received by the system. This may be considered apoint of determinism in the exchange computer system 100 that definesthe earliest point where the system can ascribe an order of receipt toarriving messages. The point of determinism may or may not be at or nearthe demarcation point between the exchange computer system 100 and apublic/internet network infrastructure. The message management module116 processes messages by interpreting the contents of a message basedon the message transmit protocol, such as the transmission controlprotocol (“TCP”), to provide the content of the message for furtherprocessing by the exchange computer system.

The message management module 116 may also be configured to detectcharacteristics of an order for a transaction to be undertaken in anelectronic marketplace. For example, the message management module 116may identify and extract order content such as a price, product, volume,and associated market participant for an order. The message managementmodule 116 may also identify and extract data indicating an action to beexecuted by the exchange computer system 100 with respect to theextracted order. For example, the message management module 116 maydetermine the transaction type of the transaction requested in a givenmessage. A message may include an instruction to perform a type oftransaction. The transaction type may be, in one embodiment, arequest/offer/order to either buy or sell a specified quantity or unitsof a financial instrument at a specified price or value. The messagemanagement module 116 may also identify and extract other orderinformation and other actions associated with the extracted order. Allextracted order characteristics, other information, and associatedactions extracted from a message for an order may be collectivelyconsidered an order as described and referenced herein.

Order or message characteristics may include, for example, the state ofthe system after a message is received, arrival time (e.g., the time amessage arrives at the MSG or Market Segment Gateway), message type(e.g., new, modify, cancel), and the number of matches generated by amessage. Order or message characteristics may also include marketparticipant side (e.g., buyer or seller) or time in force (e.g., a gooduntil end of day order that is good for the full trading day, a gooduntil canceled ordered that rests on the order book until matched, or afill or kill order that is canceled if not filled immediately, or a filland kill order (FOK) that is filled to the maximum amount possible, andany remaining or unfilled/unsatisfied quantity is not stored on thebooks or allowed to rest).

An order processing module 118 may be included to decompose delta-based,spread instrument, bulk and other types of composite orders forprocessing by the order book module 110 and/or the match engine module106. The order processing module 118 may also be used to implement oneor more procedures related to clearing an order. The order may becommunicated from the message management module 118 to the orderprocessing module 118. The order processing module 118 may be configuredto interpret the communicated order, and manage the ordercharacteristics, other information, and associated actions as they areprocessed through an order book module 110 and eventually transacted onan electronic market. For example, the order processing module 118 maystore the order characteristics and other content and execute theassociated actions. In an embodiment, the order processing module mayexecute an associated action of placing the order into an order book foran electronic trading system managed by the order book module 110. In anembodiment, placing an order into an order book and/or into anelectronic trading system may be considered a primary action for anorder. The order processing module 118 may be configured in variousarrangements, and may be configured as part of the order book module110, part of the message management module 118, or as an independentfunctioning module.

As an intermediary to electronic trading transactions, the exchangebears a certain amount of risk in each transaction that takes place. Tothat end, the clearing house implements risk management mechanisms toprotect the exchange. One or more of the modules of the exchangecomputer system 100 may be configured to determine settlement prices forconstituent contracts, such as deferred month contracts, of spreadinstruments, such as for example, settlement module 120. A settlementmodule 120 (or settlement processor or other payment processor) may beincluded to provide one or more functions related to settling orotherwise administering transactions cleared by the exchange. Settlementmodule 120 of the exchange computer system 100 may implement one or moresettlement price determination techniques. Settlement-related functionsneed not be limited to actions or events occurring at the end of acontract term. For instance, in some embodiments, settlement-relatedfunctions may include or involve daily or other mark to marketsettlements for margining purposes. In some cases, the settlement module120 may be configured to communicate with the trade database 108 (or thememory(ies) on which the trade database 108 is stored) and/or todetermine a payment amount based on a spot price, the price of thefutures contract or other financial instrument, or other price data, atvarious times. The determination may be made at one or more points intime during the term of the financial instrument in connection with amargining mechanism. For example, the settlement module 120 may be usedto determine a mark to market amount on a daily basis during the term ofthe financial instrument. Such determinations may also be made on asettlement date for the financial instrument for the purposes of finalsettlement.

In some embodiments, the settlement module 120 may be integrated to anydesired extent with one or more of the other modules or processors ofthe exchange computer system 100. For example, the settlement module 120and the risk management module 114 may be integrated to any desiredextent. In some cases, one or more margining procedures or other aspectsof the margining mechanism(s) may be implemented by the settlementmodule 120.

One or more of the above-described modules of the exchange computersystem 100 may be used to gather or obtain data to support thesettlement price determination, as well as a subsequent marginrequirement determination. For example, the order book module 110 and/orthe market data module 112 may be used to receive, access, or otherwiseobtain market data, such as bid-offer values of orders currently on theorder books. The trade database 108 may be used to receive, access, orotherwise obtain trade data indicative of the prices and volumes oftrades that were recently executed in a number of markets. In somecases, transaction data (and/or bid/ask data) may be gathered orobtained from open outcry pits and/or other sources and incorporatedinto the trade and market data from the electronic trading system(s).

It should be appreciated that concurrent processing limits may bedefined by or imposed separately or in combination on one or more of thetrading system components, including the user database 102, the accountdata module 104, the match engine module 106, the trade database 108,the order book module 110, the market data module 112, the riskmanagement module 114, the message management module 116, the orderprocessing module 118, the settlement module 120, or other component ofthe exchange computer system 100.

The disclosed mechanisms may be implemented at any logical and/orphysical point(s), or combinations thereof, at which the relevantinformation/data (e.g., message traffic and responses thereto) may bemonitored or flows or is otherwise accessible or measurable, includingone or more gateway devices, modems, the computers or terminals of oneor more market participants, e.g., client computers, etc.

One skilled in the art will appreciate that one or more modulesdescribed herein may be implemented using, among other things, atangible computer-readable medium comprising computer-executableinstructions (e.g., executable software code). Alternatively, modulesmay be implemented as software code, firmware code, specificallyconfigured hardware or processors, and/or a combination of theaforementioned. For example, the modules may be embodied as part of anexchange 100 for financial instruments. It should be appreciated thedisclosed embodiments may be implemented as a different or separatemodule of the exchange computer system 100, or a separate computersystem coupled with the exchange computer system 100 so as to haveaccess to margin account record, pricing, and/or other data. Asdescribed herein, the disclosed embodiments may be implemented as acentrally accessible system or as a distributed system, e.g., where someof the disclosed functions are performed by the computer systems of themarket participants.

The trading network environment shown in FIG. 1 includes exemplarycomputer devices 150, 152, 154, 156 and 158 which depict differentexemplary methods or media by which a computer device may be coupledwith the exchange computer system 100 or by which a user maycommunicate, e.g., send and receive, trade or other informationtherewith. It should be appreciated that the types of computer devicesdeployed by traders and the methods and media by which they communicatewith the exchange computer system 100 is implementation dependent andmay vary and that not all of the depicted computer devices and/ormeans/media of communication may be used and that other computer devicesand/or means/media of communications, now available or later developedmay be used. Each computer device, which may comprise a computer 200described in more detail with respect to FIG. 2, may include a centralprocessor, specifically configured or otherwise, that controls theoverall operation of the computer and a system bus that connects thecentral processor to one or more conventional components, such as anetwork card or modem. Each computer device may also include a varietyof interface units and drives for reading and writing data or files andcommunicating with other computer devices and with the exchange computersystem 100. Depending on the type of computer device, a user caninteract with the computer with a keyboard, pointing device, microphone,pen device or other input device now available or later developed.

An exemplary computer device 150 is shown directly connected to exchangecomputer system 100, such as via a T1 line, a common local area network(LAN) or other wired and/or wireless medium for connecting computerdevices, such as the network 220 shown in FIG. 2 and described withrespect thereto. The exemplary computer device 150 is further shownconnected to a radio 168. The user of radio 168, which may include acellular telephone, smart phone, or other wireless proprietary and/ornon-proprietary device, may be a trader or exchange employee. The radiouser may transmit orders or other information to the exemplary computerdevice 150 or a user thereof. The user of the exemplary computer device150, or the exemplary computer device 150 alone and/or autonomously, maythen transmit the trade or other information to the exchange computersystem 100.

Exemplary computer devices 152 and 154 are coupled with a local areanetwork (“LAN”) 160 which may be configured in one or more of thewell-known LAN topologies, e.g., star, daisy chain, etc., and may use avariety of different protocols, such as Ethernet, TCP/IP, etc. Theexemplary computer devices 152 and 154 may communicate with each otherand with other computer and other devices which are coupled with the LAN160. Computer and other devices may be coupled with the LAN 160 viatwisted pair wires, coaxial cable, fiber optics or other wired orwireless media. As shown in FIG. 1, an exemplary wireless personaldigital assistant device (“PDA”) 158, such as a mobile telephone, tabletbased compute device, or other wireless device, may communicate with theLAN 160 and/or the Internet 162 via radio waves, such as via WiFi,Bluetooth and/or a cellular telephone based data communicationsprotocol. PDA 158 may also communicate with exchange computer system 100via a conventional wireless hub 164.

FIG. 1 also shows the LAN 160 coupled with a wide area network (“WAN”)162 which may be comprised of one or more public or private wired orwireless networks. In one embodiment, the WAN 162 includes the Internet162. The LAN 160 may include a router to connect LAN 160 to the Internet162. Exemplary computer device 156 is shown coupled directly to theInternet 162, such as via a modem, DSL line, satellite dish or any otherdevice for connecting a computer device to the Internet 162 via aservice provider therefore as is known. LAN 160 and/or WAN 162 may bethe same as the network 220 shown in FIG. 2 and described with respectthereto.

Users of the exchange computer system 100 may include one or more marketmakers 166 which may maintain a market by providing constant bid andoffer prices for a derivative or security to the exchange computersystem 100, such as via one of the exemplary computer devices depicted.The exchange computer system 100 may also exchange information withother match or trade engines, such as trade engine 170. One skilled inthe art will appreciate that numerous additional computers and systemsmay be coupled to exchange computer system 100. Such computers andsystems may include clearing, regulatory and fee systems.

The operations of computer devices and systems shown in FIG. 1 may becontrolled by computer-executable instructions stored on anon-transitory computer-readable medium. For example, the exemplarycomputer device 152 may store computer-executable instructions forreceiving order information from a user, transmitting that orderinformation to exchange computer system 100 in electronic messages,extracting the order information from the electronic messages, executingactions relating to the messages, and/or calculating values fromcharacteristics of the extracted order to facilitate matching orders andexecuting trades. In another example, the exemplary computer device 154may include computer-executable instructions for receiving market datafrom exchange computer system 100 and displaying that information to auser.

Numerous additional servers, computers, handheld devices, personaldigital assistants, telephones and other devices may also be connectedto exchange computer system 100. Moreover, one skilled in the art willappreciate that the topology shown in FIG. 1 is merely an example andthat the components shown in FIG. 1 may include other components notshown and be connected by numerous alternative topologies.

Referring now to FIG. 2, an illustrative embodiment of a generalcomputer system 200 is shown. The computer system 200 can include a setof instructions that can be executed to cause the computer system 200 toperform any one or more of the methods or computer based functionsdisclosed herein. The computer system 200 may operate as a standalonedevice or may be connected, e.g., using a network, to other computersystems or peripheral devices. Any of the components discussed herein,such as processor 202, may be a computer system 200 or a component inthe computer system 200. The computer system 200 may be specificallyconfigured to implement a match engine, margin processing, payment orclearing function on behalf of an exchange, such as the ChicagoMercantile Exchange, of which the disclosed embodiments are a componentthereof.

In a networked deployment, the computer system 200 may operate in thecapacity of a server or as a client user computer in a client-serveruser network environment, or as a peer computer system in a peer-to-peer(or distributed) network environment. The computer system 200 can alsobe implemented as or incorporated into various devices, such as apersonal computer (PC), a tablet PC, a set-top box (STB), a personaldigital assistant (PDA), a mobile device, a palmtop computer, a laptopcomputer, a desktop computer, a communications device, a wirelesstelephone, a land-line telephone, a control system, a camera, a scanner,a facsimile machine, a printer, a pager, a personal trusted device, aweb appliance, a network router, switch or bridge, or any other machinecapable of executing a set of instructions (sequential or otherwise)that specify actions to be taken by that machine. In a particularembodiment, the computer system 200 can be implemented using electronicdevices that provide voice, video or data communication. Further, whilea single computer system 200 is illustrated, the term “system” shallalso be taken to include any collection of systems or sub-systems thatindividually or jointly execute a set, or multiple sets, of instructionsto perform one or more computer functions.

As illustrated in FIG. 2, the computer system 200 may include aprocessor 202, e.g., a central processing unit (CPU), a graphicsprocessing unit (GPU), or both. The processor 202 may be a component ina variety of systems. For example, the processor 202 may be part of astandard personal computer or a workstation. The processor 202 may beone or more general processors, digital signal processors, specificallyconfigured processors, application specific integrated circuits, fieldprogrammable gate arrays, servers, networks, digital circuits, analogcircuits, combinations thereof, or other now known or later developeddevices for analyzing and processing data. The processor 202 mayimplement a software program, such as code generated manually (i.e.,programmed).

The computer system 200 may include a memory 204 that can communicatevia a bus 208. The memory 204 may be a main memory, a static memory, ora dynamic memory. The memory 204 may include, but is not limited to,computer readable storage media such as various types of volatile andnon-volatile storage media, including but not limited to random accessmemory, read-only memory, programmable read-only memory, electricallyprogrammable read-only memory, electrically erasable read-only memory,flash memory, magnetic tape or disk, optical media and the like. In oneembodiment, the memory 204 includes a cache or random access memory forthe processor 202. In alternative embodiments, the memory 204 isseparate from the processor 202, such as a cache memory of a processor,the system memory, or other memory. The memory 204 may be an externalstorage device or database for storing data. Examples include a harddrive, compact disc (“CD”), digital video disc (“DVD”), memory card,memory stick, floppy disc, universal serial bus (“USB”) memory device,or any other device operative to store data. The memory 204 is operableto store instructions executable by the processor 202. The functions,acts or tasks illustrated in the figures or described herein may beperformed by the programmed processor 202 executing the instructions 212stored in the memory 204. The functions, acts or tasks are independentof the particular type of instructions set, storage media, processor orprocessing strategy and may be performed by software, hardware,integrated circuits, firm-ware, micro-code and the like, operating aloneor in combination. Likewise, processing strategies may includemultiprocessing, multitasking, parallel processing and the like.

As shown, the computer system 200 may further include a display unit214, such as a liquid crystal display (LCD), an organic light emittingdiode (OLED), a flat panel display, a solid state display, a cathode raytube (CRT), a projector, a printer or other now known or later developeddisplay device for outputting determined information. The display 214may act as an interface for the user to see the functioning of theprocessor 202, or specifically as an interface with the software storedin the memory 204 or in the drive unit 206.

Additionally, the computer system 200 may include an input device 216configured to allow a user to interact with any of the components ofsystem 200. The input device 216 may be a number pad, a keyboard, or acursor control device, such as a mouse, or a joystick, touch screendisplay, remote control or any other device operative to interact withthe system 200.

In a particular embodiment, as depicted in FIG. 2, the computer system200 may also include a disk or optical drive unit 206. The disk driveunit 206 may include a computer-readable medium 210 in which one or moresets of instructions 212, e.g., software, can be embedded. Further, theinstructions 212 may embody one or more of the methods or logic asdescribed herein. In a particular embodiment, the instructions 212 mayreside completely, or at least partially, within the memory 204 and/orwithin the processor 202 during execution by the computer system 200.The memory 204 and the processor 202 also may include computer-readablemedia as discussed herein.

The present disclosure contemplates a computer-readable medium thatincludes instructions 212 or receives and executes instructions 212responsive to a propagated signal, so that a device connected to anetwork 220 can communicate voice, video, audio, images or any otherdata over the network 220. Further, the instructions 212 may betransmitted or received over the network 220 via a communicationinterface 218. The communication interface 218 may be a part of theprocessor 202 or may be a separate component. The communicationinterface 218 may be created in software or may be a physical connectionin hardware. The communication interface 218 is configured to connectwith a network 220, external media, the display 214, or any othercomponents in system 200, or combinations thereof. The connection withthe network 220 may be a physical connection, such as a wired Ethernetconnection or may be established wirelessly. Likewise, the additionalconnections with other components of the system 200 may be physicalconnections or may be established wirelessly.

The network 220 may include wired networks, wireless networks, orcombinations thereof. The wireless network may be a cellular telephonenetwork, an 802.11, 802.16, 802.20, or WiMax network. Further, thenetwork 220 may be a public network, such as the Internet, a privatenetwork, such as an intranet, or combinations thereof, and may utilize avariety of networking protocols now available or later developedincluding, but not limited to, TCP/IP based networking protocols.

Embodiments of the subject matter and the functional operationsdescribed in this specification can be implemented in digital electroniccircuitry, or in computer software, firmware, or hardware, including thestructures disclosed in this specification and their structuralequivalents, or in combinations of one or more of them. Embodiments ofthe subject matter described in this specification can be implemented asone or more computer program products, i.e., one or more modules ofcomputer program instructions encoded on a computer readable medium forexecution by, or to control the operation of, data processing apparatus.While the computer-readable medium is shown to be a single medium, theterm “computer-readable medium” includes a single medium or multiplemedia, such as a centralized or distributed database, and/or associatedcaches and servers that store one or more sets of instructions. The term“computer-readable medium” shall also include any medium that is capableof storing, encoding or carrying a set of instructions for execution bya processor or that cause a computer system to perform any one or moreof the methods or operations disclosed herein. The computer readablemedium can be a machine-readable storage device, a machine-readablestorage substrate, a memory device, or a combination of one or more ofthem. The term “data processing apparatus” encompasses all apparatus,devices, and machines for processing data, including by way of example aprogrammable processor, a computer, or multiple processors or computers.The apparatus can include, in addition to hardware, code that creates anexecution environment for the computer program in question, e.g., codethat constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, or a combination of one or moreof them.

In a particular non-limiting, exemplary embodiment, thecomputer-readable medium can include a solid-state memory such as amemory card or other package that houses one or more non-volatileread-only memories. Further, the computer-readable medium can be arandom access memory or other volatile re-writable memory. Additionally,the computer-readable medium can include a magneto-optical or opticalmedium, such as a disk or tapes or other storage device to capturecarrier wave signals such as a signal communicated over a transmissionmedium. A digital file attachment to an e-mail or other self-containedinformation archive or set of archives may be considered a distributionmedium that is a tangible storage medium. Accordingly, the disclosure isconsidered to include any one or more of a computer-readable medium or adistribution medium and other equivalents and successor media, in whichdata or instructions may be stored.

In an alternative embodiment, dedicated or otherwise specificallyconfigured hardware implementations, such as application specificintegrated circuits, programmable logic arrays and other hardwaredevices, can be constructed to implement one or more of the methodsdescribed herein. Applications that may include the apparatus andsystems of various embodiments can broadly include a variety ofelectronic and computer systems. One or more embodiments describedherein may implement functions using two or more specific interconnectedhardware modules or devices with related control and data signals thatcan be communicated between and through the modules, or as portions ofan application-specific integrated circuit. Accordingly, the presentsystem encompasses software, firmware, and hardware implementations.

In accordance with various embodiments of the present disclosure, themethods described herein may be implemented by software programsexecutable by a computer system. Further, in an exemplary, non-limitedembodiment, implementations can include distributed processing,component/object distributed processing, and parallel processing.Alternatively, virtual computer system processing can be constructed toimplement one or more of the methods or functionality as describedherein.

Although the present specification describes components and functionsthat may be implemented in particular embodiments with reference toparticular standards and protocols, the invention is not limited to suchstandards and protocols. For example, standards for Internet and otherpacket switched network transmission (e.g., TCP/IP, UDP/IP, HTML, HTTP,HTTPS) represent examples of the state of the art. Such standards areperiodically superseded by faster or more efficient equivalents havingessentially the same functions. Accordingly, replacement standards andprotocols having the same or similar functions as those disclosed hereinare considered equivalents thereof.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, and it can bedeployed in any form, including as a standalone program or as a module,component, subroutine, or other unit suitable for use in a computingenvironment. A computer program does not necessarily correspond to afile in a file system. A program can be stored in a portion of a filethat holds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andanyone or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. The essential elements of a computer area processor for performing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. However, a computerneed not have such devices. Moreover, a computer can be embedded inanother device, e.g., a mobile telephone, a personal digital assistant(PDA), a mobile audio player, a Global Positioning System (GPS)receiver, to name just a few. Computer readable media suitable forstoring computer program instructions and data include all forms ofnon-volatile memory, media and memory devices, including by way ofexample semiconductor memory devices, e.g., EPROM, EEPROM, and flashmemory devices; magnetic disks, e.g., internal hard disks or removabledisks; magneto optical disks; and CD ROM and DVD-ROM disks. Theprocessor and the memory can be supplemented by, or incorporated in,special purpose logic circuitry.

As used herein, the terms “microprocessor” or “general-purposeprocessor” (“GPP”) may refer to a hardware device that fetchesinstructions and data from a memory or storage device and executes thoseinstructions (for example, an Intel Xeon processor or an AMD Opteronprocessor) to then, for example, process the data in accordancetherewith. The term “reconfigurable logic” may refer to any logictechnology whose form and function can be significantly altered (i.e.,reconfigured) in the field post-manufacture as opposed to amicroprocessor, whose function can change post-manufacture, e.g. viacomputer executable software code, but whose form, e.g. thearrangement/layout and interconnection of logical structures, is fixedat manufacture. The term “software” may refer to data processingfunctionality that is deployed on a GPP. The term “firmware” may referto data processing functionality that is deployed on reconfigurablelogic. One example of a reconfigurable logic is a field programmablegate array (“FPGA”) which is a reconfigurable integrated circuit. AnFPGA may contain programmable logic components called “logic blocks”,and a hierarchy of reconfigurable interconnects that allow the blocks tobe “wired together”, somewhat like many (changeable) logic gates thatcan be inter-wired in (many) different configurations. Logic blocks maybe configured to perform complex combinatorial functions, or merelysimple logic gates like AND, OR, NOT and XOR. An FPGA may furtherinclude memory elements, which may be simple flip-flops or more completeblocks of memory.

To provide for interaction with a user, embodiments of the subjectmatter described in this specification can be implemented on a devicehaving a display, e.g., a CRT (cathode ray tube) or LCD (liquid crystaldisplay) monitor, for displaying information to the user and a keyboardand a pointing device, e.g., a mouse or a trackball, by which the usercan provide input to the computer. Other kinds of devices can be used toprovide for interaction with a user as well. Feedback provided to theuser can be any form of sensory feedback, e.g., visual feedback,auditory feedback, or tactile feedback. Input from the user can bereceived in any form, including acoustic, speech, or tactile input.

Embodiments of the subject matter described in this specification can beimplemented in a computing system that includes a back end component,e.g., a data server, or that includes a middleware component, e.g., anapplication server, or that includes a front end component, e.g., aclient computer having a graphical user interface or a Web browserthrough which a user can interact with an implementation of the subjectmatter described in this specification, or any combination of one ormore such back end, middleware, or front end components. The componentsof the system can be interconnected by any form or medium of digitaldata communication, e.g., a communication network. Examples ofcommunication networks include a local area network (“LAN”) and a widearea network (“WAN”), e.g., the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

It should be appreciated that the disclosed embodiments may beapplicable to other types of messages depending upon the implementation.Further, the messages may comprise one or more data packets, datagramsor other collection of data formatted, arranged configured and/orpackaged in a particular one or more protocols, e.g., the FIX protocol,TCP/IP, Ethernet, etc., suitable for transmission via a network 214 aswas described, such as the message format and/or protocols described inU.S. Pat. No. 7,831,491 and U.S. Patent Publication No. 2005/0096999 A1,both of which are incorporated by reference herein in their entiretiesand relied upon. Further, the disclosed message management system may beimplemented using an open message standard implementation, such as FIX,FIX Binary, FIX/FAST, or by an exchange-provided API.

The embodiments described herein utilize trade related electronicmessages such as mass quote messages, individual order messages,modification messages, cancellation messages, etc., so as to enacttrading activity in an electronic market. The trading entity and/ormarket participant may have one or multiple trading terminals associatedwith the session. Furthermore, the financial instruments may befinancial derivative products. Derivative products may include futurescontracts, options on futures contracts, futures contracts that arefunctions of or related to other futures contracts, swaps, swaptions, orother financial instruments that have their price related to or derivedfrom an underlying product, security, commodity, equity, index, orinterest rate product. In one embodiment, the orders are for optionscontracts that belong to a common option class. Orders may also be forbaskets, quadrants, other combinations of financial instruments, etc.The option contracts may have a plurality of strike prices and/orcomprise put and call contracts. A mass quote message may be received atan exchange. As used herein, an exchange computing system 100 includes aplace or system that receives and/or executes orders.

In an embodiment, a plurality of electronic messages is received fromthe network. The plurality of electronic messages may be received at anetwork interface for the electronic trading system. The plurality ofelectronic messages may be sent from market participants. The pluralityof messages may include order characteristics and be associated withactions to be executed with respect to an order that may be extractedfrom the order characteristics. The action may involve any action asassociated with transacting the order in an electronic trading system.The actions may involve placing the orders within a particular marketand/or order book of a market in the electronic trading system.

The order processing module 118 may also store data indicative ofcharacteristics of the extracted orders. For example, the orderprocessing module may store data indicative of orders having anassociated modify or cancel action, such as by recording a count of thenumber of such orders associated with particular market participants.The order processing module may also store data indicative of quantitiesand associated prices of orders to buy or sell a product placed in themarket order book 110, as associated with particular marketparticipants.

Also, the order processing module 118 may be configured to calculate andassociate with particular orders a value indicative of an associatedmarket participant's market activity quality, which is a valueindicative of whether the market participant's market activity increasesor tends to increase liquidity of a market. This value may be determinedbased on the price of the particular order, previously stored quantitiesof orders from the associated market participant, the previously storeddata indicative of previously received orders to modify or cancel asassociated with the market participant, and previously stored dataindicative of a result of the attempt to match previously receivedorders stored in association with the market participant. The orderprocessing module 118 may determine or otherwise calculate scoresindicative of the quality value based on these stored extracted ordercharacteristics, such as an MQI as described herein.

Further, electronic trading systems may perform actions on orders placedfrom received messages based on various characteristics of the messagesand/or market participants associated with the messages. These actionsmay include matching the orders either during a continuous auctionprocess, or at the conclusion of a collection period during a batchauction process. The matching of orders may be by any technique.

The matching of orders may occur based on a priority indicated by thecharacteristics of orders and market participants associated with theorders. Orders having a higher priority may be matched before orders ofa lower priority. Such priority may be determined using varioustechniques. For example, orders that were indicated by messages receivedearlier may receive a higher priority to match than orders that wereindicated by messages received later. Also, scoring or grading of thecharacteristics may provide for priority determination. Data indicativeof order matches may be stored by a match engine and/or an orderprocessing module 118, and used for determining MQI scores of marketparticipants.

Example Users

Generally, a market may involve market makers, such as marketparticipants who consistently provide bids and/or offers at specificprices in a manner typically conducive to balancing risk, and markettakers who may be willing to execute transactions at prevailing bids oroffers may be characterized by more aggressive actions so as to maintainrisk and/or exposure as a speculative investment strategy. From analternate perspective, a market maker may be considered a marketparticipant who places an order to sell at a price at which there is nopreviously or concurrently provided counter order. Similarly, a markettaker may be considered a market participant who places an order to buyat a price at which there is a previously or concurrently providedcounter order. A balanced and efficient market may involve both marketmakers and market takers, coexisting in a mutually beneficial basis. Themutual existence, when functioning properly, may facilitate liquidity inthe market such that a market may exist with “tight” bid-ask spreads(e.g., small difference between bid and ask prices) and a “deep” volumefrom many currently provided orders such that large quantity orders maybe executed without driving prices significantly higher or lower.

As such, both market participant types are useful in generatingliquidity in a market, but specific characteristics of market activitytaken by market participants may provide an indication of a particularmarket participant's effect on market liquidity. For example, a MarketQuality Index (“MQI”) of an order may be determined using thecharacteristics. An MQI may be considered a value indicating alikelihood that a particular order will improve or facilitate liquidityin a market. That is, the value may indicate a likelihood that the orderwill increase a probability that subsequent requests and transactionfrom other market participants will be satisfied. As such, an MQI may bedetermined based on a proximity of the entered price of an order to amidpoint of a current bid-ask price spread, a size of the entered order,a volume or quantity of previously filled orders of the marketparticipant associated with the order, and/or a frequency ofmodifications to previous orders of the market participant associatedwith the order. In this way, an electronic trading system may functionto assess and/or assign an MQI to received electronic messages toestablish messages that have a higher value to the system, and thus thesystem may use computing resources more efficiently by expendingresources to match orders of the higher value messages prior toexpending resources of lower value messages.

While an MQI may be applied to any or all market participants, such anindex may also be applied only to a subset thereof, such as large marketparticipants, or market participants whose market activity as measuredin terms of average daily message traffic over a limited historical timeperiod exceeds a specified number. For example, a market participantgenerating more than 500, 1,000, or even 10,000 market messages per daymay be considered a large market participant.

An exchange provides one or more markets for the purchase and sale ofvarious types of products including financial instruments such asstocks, bonds, futures contracts, options, currency, cash, and othersimilar instruments. Agricultural products and commodities are alsoexamples of products traded on such exchanges. A futures contract is aproduct that is a contract for the future delivery of another financialinstrument such as a quantity of grains, metals, oils, bonds, currency,or cash. Generally, each exchange establishes a specification for eachmarket provided thereby that defines at least the product traded in themarket, minimum quantities that must be traded, and minimum changes inprice (e.g., tick size). For some types of products (e.g., futures oroptions), the specification further defines a quantity of the underlyingproduct represented by one unit (or lot) of the product, and deliveryand expiration dates. As will be described, the exchange may furtherdefine the matching algorithm, or rules, by which incoming orders willbe matched/allocated to resting orders.

Matching and Transaction Processing

Market participants, e.g., traders, use software to send orders ormessages to the trading platform. The order identifies the product, thequantity of the product the trader wishes to trade, a price at which thetrader wishes to trade the product, and a direction of the order (i.e.,whether the order is a bid, i.e., an offer to buy, or an ask, i.e., anoffer to sell). It will be appreciated that there may be other ordertypes or messages that traders can send including requests to modify orcancel a previously submitted order.

The exchange computer system monitors incoming orders received therebyand attempts to identify, i.e., match or allocate, as described herein,one or more previously received, but not yet matched, orders, i.e.,limit orders to buy or sell a given quantity at a given price, referredto as “resting” orders, stored in an order book database, wherein eachidentified order is contra to the incoming order and has a favorableprice relative to the incoming order. An incoming order may be an“aggressor” order, i.e., a market order to sell a given quantity atwhatever may be the current resting bid order price(s) or a market orderto buy a given quantity at whatever may be the current resting ask orderprice(s). An incoming order may be a “market making” order, i.e., amarket order to buy or sell at a price for which there are currently noresting orders. In particular, if the incoming order is a bid, i.e., anoffer to buy, then the identified order(s) will be an ask, i.e., anoffer to sell, at a price that is identical to or higher than the bidprice. Similarly, if the incoming order is an ask, i.e., an offer tosell, the identified order(s) will be a bid, i.e., an offer to buy, at aprice that is identical to or lower than the offer price.

An exchange computing system may receive conditional orders or messagesfor a data object, where the order may include two prices or values: areference value and a stop value. A conditional order may be configuredso that when a product represented by the data object trades at thereference price, the stop order is activated at the stop value. Forexample, if the exchange computing system's order management moduleincludes a stop order with a stop price of 5 and a limit price of 1 fora product, and a trade at 5 (i.e., the stop price of the stop order)occurs, then the exchange computing system attempts to trade at 1 (i.e.,the limit price of the stop order). In other words, a stop order is aconditional order to trade (or execute) at the limit price that istriggered (or elected) when a trade at the stop price occurs.

Stop orders also rest on, or are maintained in, an order book to monitorfor a trade at the stop price, which triggers an attempted trade at thelimit price. In some embodiments, a triggered limit price for a stoporder may be treated as an incoming order.

Upon identification (matching) of a contra order(s), a minimum of thequantities associated with the identified order and the incoming orderis matched and that quantity of each of the identified and incomingorders become two halves of a matched trade that is sent to a clearinghouse. The exchange computer system considers each identified order inthis manner until either all of the identified orders have beenconsidered or all of the quantity associated with the incoming order hasbeen matched, i.e., the order has been filled. If any quantity of theincoming order remains, an entry may be created in the order bookdatabase and information regarding the incoming order is recordedtherein, i.e., a resting order is placed on the order book for theremaining quantity to await a subsequent incoming order counter thereto.

It should be appreciated that in electronic trading systems implementedvia an exchange computing system, a trade price (or match value) maydiffer from (i.e., be better for the submitter, e.g., lower than asubmitted buy price or higher than a submitted sell price) the limitprice that is submitted, e.g., a price included in an incoming message,or a triggered limit price from a stop order.

As used herein, “better” than a reference value means lower than thereference value if the transaction is a purchase (or acquire)transaction, and higher than the reference value if the transaction is asell transaction. Said another way, for purchase (or acquire)transactions, lower values are better, and for relinquish or selltransactions, higher values are better.

Traders access the markets on a trading platform using trading softwarethat receives and displays at least a portion of the order book for amarket, i.e., at least a portion of the currently resting orders,enables a trader to provide parameters for an order for the producttraded in the market, and transmits the order to the exchange computersystem. The trading software typically includes a graphical userinterface to display at least a price and quantity of some of theentries in the order book associated with the market. The number ofentries of the order book displayed is generally preconfigured by thetrading software, limited by the exchange computer system, or customizedby the user. Some graphical user interfaces display order books ofmultiple markets of one or more trading platforms. The trader may be anindividual who trades on his/her behalf, a broker trading on behalf ofanother person or entity, a group, or an entity. Furthermore, the tradermay be a system that automatically generates and submits orders.

If the exchange computer system identifies that an incoming market ordermay be filled by a combination of multiple resting orders, e.g., theresting order at the best price only partially fills the incoming order,the exchange computer system may allocate the remaining quantity of theincoming, i.e., that which was not filled by the resting order at thebest price, among such identified orders in accordance withprioritization and allocation rules/algorithms, referred to as“allocation algorithms” or “matching algorithms,” as, for example, maybe defined in the specification of the particular financial product ordefined by the exchange for multiple financial products. Similarly, ifthe exchange computer system identifies multiple orders contra to theincoming limit order and that have an identical price which is favorableto the price of the incoming order, i.e., the price is equal to orbetter, e.g., lower if the incoming order is a buy (or instruction topurchase, or instruction to acquire) or higher if the incoming order isa sell (or instruction to relinquish), than the price of the incomingorder, the exchange computer system may allocate the quantity of theincoming order among such identified orders in accordance with thematching algorithms as, for example, may be defined in the specificationof the particular financial product or defined by the exchange formultiple financial products.

An exchange responds to inputs, such as trader orders, cancellation,etc., in a manner as expected by the market participants, such as basedon market data, e.g., prices, available counter-orders, etc., to providean expected level of certainty that transactions will occur in aconsistent and predictable manner and without unknown or unascertainablerisks. Accordingly, the method by which incoming orders are matched withresting orders must be defined so that market participants have anexpectation of what the result will be when they place an order or haveresting orders and an incoming order is received, even if the expectedresult is, in fact, at least partially unpredictable due to somecomponent of the process being random or arbitrary or due to marketparticipants having imperfect or less than all information, e.g.,unknown position of an order in an order book. Typically, the exchangedefines the matching/allocation algorithm that will be used for aparticular financial product, with or without input from the marketparticipants. Once defined for a particular product, thematching/allocation algorithm is typically not altered, except inlimited circumstance, such as to correct errors or improve operation, soas not to disrupt trader expectations. It will be appreciated thatdifferent products offered by a particular exchange may use differentmatching algorithms.

One exemplary system for matching is described in U.S. patentapplication Ser. No. 13/534,499, filed on Jun. 27, 2012, entitled“Multiple Trade Matching Algorithms,” published as U.S. PatentApplication Publication No. 2014/0006243 A1, the entirety of which isincorporated by reference herein and relied upon.

Spread Instruments

Traders trading on an exchange including, for example, exchange computersystem 100, often desire to trade multiple financial instruments incombination. Each component of the combination may be called a leg.Traders can submit orders for individual legs or in some cases cansubmit a single order for multiple financial instruments in anexchange-defined combination. Such orders may be called a strategyorder, a spread order, or a variety of other names.

A spread instrument may involve the simultaneous purchase of onesecurity and sale of a related security, called legs, as a unit. Thelegs of a spread instrument may be options or futures contracts, orcombinations of the two. Trades in spread instruments are executed toyield an overall net position whose value, called the spread, depends onthe difference between the prices of the legs. Spread instruments may betraded in an attempt to profit from the widening or narrowing of thespread, rather than from movement in the prices of the legs directly.Spread instruments are either “bought” or “sold” depending on whetherthe trade will profit from the widening or narrowing of the spread,respectively. An exchange often supports trading of common spreads as aunit rather than as individual legs, thus ensuring simultaneousexecution of the two legs, eliminating the execution risk of one legexecuting but the other failing.

Implication

Thus an exchange may match outright orders, such as individual contractsor spread orders (which as discussed herein could include multipleindividual contracts). The exchange may also imply orders from outrightorders. For example, exchange computer system 100 may derive, identifyand/or advertise, publish, display or otherwise make available fortrading orders based on outright orders.

As was described above, the financial instruments which are the subjectof the orders to trade, may include one or more component financialinstruments. While each financial instrument may have its own orderbook, i.e. market, in which it may be traded, in the case of a financialinstrument having more than one component financial instrument, thosecomponent financial instruments may further have their own order booksin which they may be traded. Accordingly, when an order for a financialinstrument is received, it may be matched against a suitable counterorder in its own order book or, possibly, against a combination ofsuitable counter orders in the order books the component financialinstruments thereof, or which share a common component financialinstrument. For example, an order for a spread contract comprisingcomponent financial instruments A and B may be matched against anothersuitable order for that spread contract. However, it may also be matchedagainst suitable separate counter orders for the A and for the Bcomponent financial instruments found in the order books therefore.Similarly, if an order for the A contract is received and suitable matchcannot be found in the A order book, it may be possible to match orderfor A against a combination of a suitable counter order for a spreadcontract comprising the A and B component financial instruments and asuitable counter order for the B component financial instrument. This isreferred to as “implication” where a given order for a financialinstrument may be matched via a combination of suitable counter ordersfor financial instruments which share common, or otherwiseinterdependent, component financial instruments. Implication increasesthe liquidity of the market by providing additional opportunities fororders to be traded. Increasing the number of transactions may furtherincrease the number of transaction fees collected by the electronictrading system.

The order for a particular financial instrument actually received from amarket participant, whether it comprises one or more component financialinstruments, is referred to as a “real” or “outright” order, or simplyas an outright. The one or more orders which must be synthesized andsubmitted into order books other than the order book for the outrightorder in order to create matches therein, are referred to as “implied”orders. Upon receipt of an incoming order, the identification orderivation of suitable implied orders which would allow at least apartial trade of the incoming outright order to be executed is referredto as “implication” or “implied matching”, the identified orders beingreferred to as an “implied match.” Depending on the number componentfinancial instruments involved, and whether those component financialinstruments further comprise component financial instruments of theirown, there may be numerous different implied matches identified whichwould allow the incoming order to be at least partially matched andmechanisms may be provided to arbitrate, e.g., automatically, amongthem, such as by picking the implied match comprising the least numberof component financial instruments or the least number of synthesizedorders.

Upon receipt of an incoming order, or thereafter, a combination of oneor more suitable/hypothetical counter orders which have not actuallybeen received but if they were received, would allow at least a partialtrade of the incoming order to be executed, may be, e.g., automatically,identified or derived and referred to as an “implied opportunity.” Aswith implied matches, there may be numerous implied opportunitiesidentified for a given incoming order. Implied opportunities areadvertised to the market participants, such as via suitable syntheticorders, e.g. counter to the desired order, being placed on therespective order books to rest (or give the appearance that there is anorder resting) and presented via the market data feed, electronicallycommunicated to the market participants, to appear available to trade inorder to solicit the desired orders from the market participants.Depending on the number component financial instruments involved, andwhether those component financial instruments further comprise componentfinancial instruments of their own, there may be numerous impliedopportunities, the submission of a counter order in response thereto,would allow the incoming order to be at least partially matched.

Implied opportunities, e.g. the advertised synthetic orders, mayfrequently have better prices than the corresponding real orders in thesame contract. This can occur when two or more traders incrementallyimprove their order prices in the hope of attracting a trade, sincecombining the small improvements from two or more real orders can resultin a big improvement in their combination. In general, advertisingimplied opportunities at better prices will encourage traders to enterthe opposing orders to trade with them. The more implied opportunitiesthat the match engine of an electronic trading system cancalculate/derive, the greater this encouragement will be and the morethe Exchange will benefit from increased transaction volume. However,identifying implied opportunities may be computationally intensive. In ahigh performance trading system where low transaction latency isimportant, it may be important to identify and advertise impliedopportunities quickly so as to improve or maintain market participantinterest and/or market liquidity.

Examples of implied spread trading include those disclosed in U.S.Patent Publication No. 2005/0203826, entitled “Implied Spread TradingSystem,” the entire disclosure of which is incorporated by referenceherein and relied upon. Examples of implied markets include thosedisclosed in U.S. Pat. No. 7,039,610, entitled “Implied Market TradingSystem,” the entire disclosure of which is incorporated by referenceherein and relied upon.

Data Storage System

The data storage system may be implemented by multiple operatingprocesses, threads, tasks or other computer program code construct,logically distributed or otherwise coupled throughout the exchangecomputer system 100 to monitor different parts, e.g. modules, thereofand record data regarding the operation thereof in a log file or otherdata store. The data store may include one or more data files, recordsor other structures or resources for storing data. As described herein,the disclosed embodiments enable multiple threads/processes to appendtheir data to the same data store, however that data store may change,e.g. a new data store may be provided once the storage capacity of thecurrent data store is reached. For example, the data store may be a datafile having a maximum capacity. As the threads/processes store data intothe data file, the capacity of the data file is monitored. Once thecapacity of the data file has been exhausted, or once it is determinedthat a thread's write request cannot fit in the available remainingcapacity of an active data store, the data file is closed and a new datafile is opened, referred to as “rolling”, and the threads/processescontinue to write their data to the new data file, as will be described.Accordingly, as used herein, the terms data store, data file, etc., mayrefer to the current data store or file, to which the multiple threadsare currently storing their data, of a set of at least one data store orfile, wherein each data store or file of the set may be created asneeded and/or created in advance.

FIG. 3 illustrates an example data storage system 300, which in oneembodiment is implemented as part of the exchange computer system 100described herein. In particular, FIG. 3 shows a system 300 forcontrolling storage, i.e. appending, of data, such as log data, in aselected one of at least one data store 301, e.g. a data file or otherdata storage construct, coupled with a processor 308. The selected oneof the at least one data store 301 may be stored in a memory 204 orelsewhere. The processor 308, memory 204 and/or data store 301 may beimplemented by a processor 202 and memory 204 as described herein withrespect to FIG. 2.

The data store 301 may be associated with an acknowledgment counter 302and an allocation counter 304. The acknowledgment counter 302 may beused to indicate the portions of the data store 301 that stores data,e.g., data has been written to the portions indicated by theacknowledgment counter 302. The allocation counter 304 may be indicativeof portions of the data store 301 that have been allocated for datastorage in response to requests to store data. Allocated portions of thedata store 301 may not yet actually contain or store data.

The system 300 further includes a requestor interface 306, which may beimplemented as a separate hardware component or as first logic 306stored in the memory 204 and executable by the processor 202 to causethe processor 202 to communicate with requestors (not shown), e.g. oneof the multiple threads which may attempt to store amounts of data,e.g., specified in bytes, in the data store 301. For example, therequestor interface 306 may receive requests to store data in the datastore 301, and may transmit information about the data store, such asthe state of the acknowledgment counter and allocation counter discussedbelow, to the one or more requestors/threads.

FIG. 4 illustrates an example flowchart of a computer implemented method400 for controlling storage of data in one or more data stores.Embodiments may involve all, more or fewer actions than the illustratedactions. The actions may be performed in the order or sequence shown, orin a different sequence.

At step 402, method 400 includes receiving one or more requests from oneor more requestors to store an amount of data.

At step 404, method 400 includes returning, to each requestor of the oneor more requestors, a value of an allocation counter associated with anactive data store, the active data store having a size and associatedwith an acknowledgment counter.

At step 406, method 400 includes updating the value of the allocationcounter based on each amount of data to be stored. For example, theallocation counter may be incremented by the amount of data to be storedin each request.

At step 408, method 400 includes determining, by each requestor, whetherthe returned value of the allocation counter does not exceed the size ofthe active data store and whether the returned value of the allocationcounter plus the amount of data requested stored by the requestorexceeds the size of the active data store. It should be appreciated thatin a multi-threaded system, multiple requestors may implement method 400concurrently. Some of the requestors may obtain different determinationresults at step 408 than other requestors. In one embodiment, becausethe data storage system updates the allocation counter in response toeach received request, and because of the arrangement and sequence ofthe steps disclosed herein, only one requestor will determine that thereturned value of the allocation counter does not exceed the size of theactive data store and that the returned value of the allocation counterplus the amount of data requested stored by the requestor exceeds thesize of the active data store.

At step 410, method 400 includes upon determining, by any one requestorof the one or more requestors, that the returned value of the allocationcounter does not exceed the size of the active data store and that thereturned value of the allocation counter plus the amount of datarequested stored by the one requestor exceeds the size of the activedata store, designating, by the one requestor, a new data store as theactive data store.

In one embodiment, method 400 may include incrementing the allocationcounter by the amount of data requested stored by a requestor beforereturning the value of the allocation counter to the requestor, wherethe incrementing and the returning are performed atomically. In such anembodiment, the requestor may determine whether the returned value ofthe allocation counter minus the amount of data requested stored by therequestor does not exceed the size of the active data store and whetherthe returned value of the allocation counter (e.g., after it has beenincremented by the amount of data requested stored by the requestor)exceeds the size of the active data store.

In one embodiment, method 400 includes causing, by the one requestor,the data store previously designated as the active data store to becleaned.

In one embodiment, method 400 includes wherein the requestors other thanthe one requestor of the one or more requestors do not designate a newdata store as the active data store or cause the data store previouslydesignated as the active data store to be cleaned. For example, the datastorage system may be configured as discussed herein so that only onerequestor designates a new data store as the active data store or causesthe data store previously designated as the active data store to becleaned,

In one embodiment, method 400 includes waiting, by the one requestor,until all the data from all the requests to store amounts of datareceived before the one requestor's request to store data has beenstored in the data store previously designated as the active data storebefore causing the data store previously designated as the active datastore to be cleaned.

In one embodiment, method 400 includes upon determining by anotherrequestor of the one or more requestors that the returned value of theallocation counter plus the amount of data requested stored by theanother requestor does not exceed the size of the active data store,commencing storing the data requested stored by the another requestor inthe active data store. For example, if a requestor determines that thereturned value of the allocation counter plus the amount of datarequested stored by that requestor does not exceed the size of theactive data store, then the active data store may not need to beswapped, and that requestor is allowed to commence storing its data tothe active data store.

In one embodiment, method 400 includes upon determining by a requestorof the one or more requestors that the returned value of the allocationcounter associated with the active data store exceeds the size of theactive data store, re-submitting, by that requestor, that requestor'spreviously submitted request to store data. For example, if a requestordetermines that the allocation counter is already past the end of thefile, then that requestor did not cause the file to become full, andthus another requestor should be responsible for swapping the datastore. Thus, the requestor that determines that the returned value ofthe allocation counter associated with the active data store exceeds thesize of the active data store waits until a new file is available wheredata can be written. In one embodiment, method 400 includes receiving,by that requestor, a returned value of an allocation counter associatedwith a new data store; and upon determining by that requestor that thereturned value of the allocation counter associated with the new datastore plus the amount of data requested stored by the another requestordoes not exceed the size of the new data store, commencing storing thedata requested stored by that requestor in the new data store. Thus, therequestor that determined that the returned value of the allocationcounter associated with the active data store exceeded the size of theactive data store and waited until a new file is available where datacan be written can now write data to the new data store, as long as thereturned value of the allocation counter associated with the new datastore plus the amount of data requested stored by that requestor doesnot exceed the size of the new data store.

FIG. 5A illustrates an example data store 500. Example data store 500has a file size of 50 bytes, as shown in FIG. 5A. The file size may beconfigurable by the system, so that the maximum file size for any givenapplication may be specified by the system. The disclosed embodimentshelp ensure that the data store file size (i.e., the amount of dataactually written to the file) does not exceed the specified maximum filesize. In the example of FIG. 5A, data is written to the data store 500and illustrated as being written from the top the bottom, e.g., from theaddress of 0 at the top to the address of 50 at the bottom.

Data store 500 is associated with an acknowledgement counter 502 andallocation counter 504. In the example of FIG. 5A, the acknowledgementcounter is at an address of 25 and the allocation counter is at anaddress of 30. The acknowledgement counter is indicative of the addresswhere data has actually been written in the data store. The allocationcounter is indicative of which portions of the data store 500 have beenallocated for data storage, even if all of those indicated portions donot yet contain/store data. Accordingly, it should be understood thatthe value of the allocation counter may be the same as the value of theacknowledgement counter but may not be less than the value of theacknowledgement counter. In other words, the data store's allocationcounter may allocate space within the data store that should be writtento in the future, but is empty at a given time. In FIG. 5A, data hasbeen written in the portion of data store 500 from the address of 0 to25, and the portion of data store 500 from the address of 0 to 30 hasbeen allocated to requestors/data write requests.

The data storage system may continue to write data to the data store500. For example, the data storage system may continue to write datafrom previously received requests. As shown in FIG. 5B, the allocationcounter 504 is still at 30 (because no additional space has beenallocated, compared to FIG. 5A), but the acknowledgement counter 502 hasmoved from 25 (FIG. 5A) to 28 (FIG. 5B) (because 3 bytes of data havebeen written to the data store compared to FIG. 5A).

The data storage system may receive requests to write data to the datastore 500. As also shown in FIG. 5B, the data storage system receives arequest 506 from requestor 0 to write 5 bytes of data.

Upon receiving the request 506, the allocation counter 504 isincremented by 5 bytes, as shown in FIG. 5C. In FIG. 5C, the allocationcounter 504 has increased to 35. The data storage system may thencontinue to write data (either from request 506, or some otherpreviously received request) to the data store 500. The acknowledgementcounter 502 may then continue to indicate where the data has beenwritten to in the data store 500. For example, in FIG. 5D, theallocation counter 504 still indicates that space up to address 35 hasbeen allocated (compared to FIG. 5C), but allocation counter 502indicates that compared to FIG. 5C, 4 more bytes have been written tothe data store (as indicated by the change from 28 to 32 for theallocation counter 502).

Data store 500 may receive additional requests to write data. Forexample, data store 500 may receive a plurality of requests from variousrequestors concurrently to write data to the same common data store 500.As shown in FIG. 5E, the system receives another request 508 fromrequestor 1 to write 17 bytes of data to data store 500. However, thedata store only has a remaining available capacity of 15 bytes (50 bytefile size minus allocation counter 504 address of 35 bytes).

Even though request 508 will not fit in the data store 500, the datastorage system increases the allocation counter 504 based on request508. As shown in FIG. 5F, the allocation counter 504 is increased by 17,to a value of 52 (i.e., 35+17=52). Thus, it should be appreciated thatthe value of the allocation counter 504 is updated based on a receivedrequest, even if the value of the allocation counter 504 exceeds thetotal file size of the data store.

Requestor 1 may run an implementation of the disclosed embodiments asdiscussed below and determine that requestor 1 is the process/threadthat should be responsible for rolling the file, as discussed below. Inparticular, the data storage system returns (not shown) a value of 35,which is the address of allocation counter 504 before receiving request508, to requestor 1. Requestor 1 uses the information about the value ofthe allocation counter 504 before the allocation counter 504 was updatedbased on the request 508 to determine whether requestor 1 should beresponsible for rolling the file. For example, requestor 1 may determinethat the allocation counter 504 was less than or equal to the file size(i.e., 32 is less than 50) before receiving requestor 1's request 508,and that the allocation counter 504 is greater than the file size (i.e.,52 is greater than 50) after receiving requestor 1 's request 508, andshould therefore be the thread/request that is responsible for swappingthe data store 500 for a new data store.

In one embodiment, the data storage system may return a value of 35 torequestor 1 and update the value of allocation counter 504 to 52atomically. For example, the data storage system may use a LOCK XADDinstruction on Intel® ×86 computer systems.

Before the data store is swapped, the system may receive anotherrequest, which may be considered concurrent to request 1, request 510from requestor 2 to write 5 bytes of data, as shown in FIG. 5G. The datastorage system may again return the current value of the allocationcounter 504, 52, to requestor 2, and atomically update the value of theallocation counter 504 by 5, to 57, as shown in FIG. 5H.

In the disclosed embodiments, even though the data store's remainingavailable capacity of 15 bytes is enough to fit or store the 5 byterequest 510, the data storage system allocates data to requests in theorder that the requests were received, and accordingly, the data storagesystem returns a value of 57 (i.e., 52+5) for the allocation counter torequestor 2.

Requestor 2 may also run an implementation of the disclosed embodimentsas discussed below and determine that requestor 2 should not beresponsible for rolling the file, as discussed below. In particular, thedata storage system returns (not shown) a value of 52, which is theaddress of allocation counter 504 before receiving request 510, torequestor 2. Requestor 2 uses the information about the value of theallocation counter 504 before the allocation counter 504 was updatedbased on the request 510 to determine whether requestor 2 should beresponsible for rolling the file. For example, requestor 2 may determinethat the allocation counter 504 was greater than the file size (i.e., 52is greater than 50) before receiving requestor 2's request 510, and thatthe allocation counter 504 is also greater than the file size (i.e., 57is greater than 50) after receiving requestor 2's request 510, andshould not therefore be responsible for swapping the data store 500 fora new data store. Requestor 2 instead determines that it should waituntil a new file is available before the data storage system canallocate space to store the 5 bytes associated with request 510. Asdiscussed above, requestor 1 will determine that it should be thethread/request that is responsible for swapping the data store 500 for anew data store.

FIG. 5I illustrates different references to the state of the system asstored by the data storage system 500, requestor 1 state 520, andrequestor 2 state 530. As can be seen, different processes/threads orcomponents may have a different overall stored reference of the state ofthe system, based on the data received by the process/thread orcomponent, when the system state was last refreshed by theprocess/thread or component, and/or implementation design, e.g., by anapplication developer.

As should be appreciated, and as shown in FIG. 5I, the actual value ofthe allocation counter 504 after the data storage system receivesrequests 508 and 510, but before the data store 500 is swapped out for anew data store, is 57.

However, as shown in FIG. 5I, after the data storage system receivesrequests 508 and 510, but before the data store 500 is swapped out for anew data store, requestor 1 state 520 has 35 as the value of theallocation counter 504 (before it is updated based on request 508).Requestor 1 can then calculate that the allocation counter 504 will beupdated to 35+17=52 based on request 508.

Also, after the data storage system receives requests 508 and 510, butbefore the data store 500 is swapped out for a new data store, requestor2 state 530 has 52 as the value of the allocation counter 504 (before itis updated based on request 510). Requestor 2 can then calculate thatthe allocation counter 504 will be updated to 52+5=57 based on request510.

Thus, a data store 500 may concurrently receive (e.g., receiving arequest before a data store is swapped, or receiving a request beforethe data storage system has completed handling a previously receivedrequest) two requests to store data, and only one thread will determinethat it should roll the file, avoiding potential conflicts that canarise from multiple threads attempting to swap the data store. Moreover,as discussed herein, requestors 1 and 2 do not need to implementCAS-type operations to allocate or write data to the data store,improving the efficiency with which a data storage system can respond toand service write requests, thereby improving the overall performance ofa computer system.

In one embodiment, a file may be force-rolled by the application ondemand (e.g., before the file is full). The disclosed embodiments allow,in many instances, a thread (e.g., 508) to roll to a new file/datastore, even if the data store is not completely full. Even if the datastore has enough capacity to store the data associated with anotherreceived request (e.g., 510), the disclosed embodiments may sacrificethe ability to write to the remaining available capacity of the datastore (e.g., the 15 unused bytes of data store 500) in order to gain thetechnical improvements and efficiencies described herein when thedisclosed data storage system is implemented. However, the disclosedembodiments do not necessarily waste the remaining available capacity ofthe data store when such capacity is unused before the file is rolled,because a thread rolls and cleans a data store only upon determiningthat the remaining available capacity is not enough to store the amountof data requested stored by the rolling thread.

As should be appreciated, the disclosed embodiments may allow multipleprocesses to cause modification of the allocation counter. For example,the data storage system may update the allocation counter in response towrite requests received concurrently.

FIG. 6 illustrates an example implementation of the data storage system300. The implementation may be a process, an instance of which isinitiated with each received request as will be described. As such,multiple instances of the depicted process may be executing at any giventime. In the depicted implementation 600, a given requestingprocess/thread which wishes to store data to a data store 301 that hassome remaining available capacity, but not enough remaining availablecapacity to store the amount of data that the given requestingprocess/thread wishes to store, is assigned the task of beingresponsible for swapping the data store 301. This decreases delays inupdating the state of data store 301, eliminates the use of instructionsthat could fail and would need to be repeated, and improves theperformance of multi-threaded write operations to a data store 301.

The implementation 600 includes an input 602, which may be implementedby the requestor interface 306, for receiving a request from arequestor, e.g. a process/thread requesting permission to write to thedata store 301. The request includes a specification of the number ofbytes the requesting process wishes to append to, or store in, the datastore 301. Upon receipt of the request, the implementation 600 obtains604 an indicator of, e.g. pointer to, the current data store 301.Implementation 600 allows for multiple data stores 301 to be in use at agiven time, where each active data store is identified by an indicator,e.g. a pointer, which identifies at least the starting location wherethe identified data store 301 is located, e.g. in a memory 204. Once thelocation of the data store 301 is known, the implementation 600 returnsthe current value of an allocation counter (e.g., a starting offset)associated with the data store 301 to the requester and also,atomically, increments 606 the allocation counter by the amount of datarequested stored by the requestor. In other words, the value for theallocation counter returned to the requestor is the value of theallocation counter before the allocation counter is incremented with theamount of data requested stored by the requestor. The implementation 600then determines 608 whether the starting offset (e.g., before theallocation counter was incremented by the amount of data requestedstored) plus the amount of data requested stored is greater than themaximum file size of the data store 301.

If the starting offset plus the amount of data requested stored is lessthan the maximum file size of the data store 301, the implementation 600allows 610 the requesting thread/process to write its data in theallocated space of the data store 301. Once the requestingthread/process has completed its storage of the data, it willacknowledge to the implementation 600 that it has completed writing,upon receipt of which, the implementation 600 will account 611 for thecumulative number of bytes of data actually written to the data store301, e.g. by accumulating or otherwise incrementing the acknowledgmentcounter by the requested number of bytes.

If the starting offset plus the amount of data requested stored isgreater than the maximum file size of the data store 301, theimplementation 600 then determines 612 whether the starting offset(e.g., before the allocation counter was incremented by the amount ofdata requested stored) is less than or equal to the maximum file size ofthe data store 301.

If the starting offset is greater than the maximum file size of the datastore 301, the implementation 600 returns to the input stage. Forexample, the implementation may attempt to write to the currently activefile again e.g., it may refresh its view of the currently active fileand may attempt to write to what it determines to be the currentlyactive file.

If the starting offset is less than or equal to the maximum file size ofthe data store 301, the implementation 600 determines that therequesting thread is the thread that should be responsible for cleaningup of the data store 301, and stores 614 the starting offset as a totalbytes allocated. Because the implementation has previously determined(at 608) that the number of bytes of data requested to be written willnot fit in the current data store 301, a new data store 301 must becreated or otherwise provided, referred to as a “roll” or “rolling thedata store”.

The implementation 600 then determines 616 if the requestor should cleanup the data store 301, or if a separate (e.g., some other) thread shouldclean up the data store 301.

If the requestor should clean up the data store 301, the implementation600 then 618 waits until the acknowledgment counter associated with thedata store 301 reaches the stored total bytes allocated, to ensure thatall the allocated data is actually written to the data store 301. Inother words, the implementation 600 checks the cumulative number ofbytes actually written (e.g., indicated by the acknowledgment counter)to the current data store 301 against the stored total bytes allocatedto determine whether any other threads/processes, previously approved tostore data, are still in the process of storing their data in thecurrent data store 301. If the cumulative number of bytes written doesnot equal the number of allocated bytes, then another thread/processmust still be storing its data and the implementation 600 waits until itis determined that all other process/threads have completed storingtheir data to the current data store 301, i.e. that the cumulativenumber of bytes written (e.g., acknowledgment counter) equals the numberof bytes allocated (e.g., the stored total bytes allocated). After theacknowledgment counter associated with the data store 301 reaches thestored total bytes allocated, the data store 301 is closed 620 and theimplementation 600 returns to the input stage.

If a separate thread, e.g., a dedicated cleaning thread, should clean upthe data store 301, the separate thread may also perform checks (notshown) similar to the checks in 618 before cleaning the file. It shouldbe appreciated that it may be advantageous to delegate cleaning a datastore to a dedicated cleaning process, so that the thread that writesdata associated with an application (e.g., requestor 1) is not sloweddown by the task of cleaning a data store.

FIG. 7 illustrates another example implementation 700 of the datastorage system 300. Similar to implementation 600, implementation 700may be a process, an instance of which is initiated with each receivedrequest as will be described. As such, multiple instances of thedepicted process 700 may be executing at any given time. In the depictedimplementation 700, unlike implementation 600, a process/threadimplements one CAS-type operation to swap old/used files. Likeimplementation 600, a process/thread in implementation 700 does notimplement CAS-type operations to allocate data, or to write data to thedata store, thereby attaining the benefits of avoiding CAS-typeoperations described herein.

The implementation 700 includes an input 702, which may be implementedby the requestor interface 306, for receiving a request from arequestor, e.g. a process/thread requesting permission to write to thedata store 301. The request includes a specification of the number ofbytes the requesting process wishes to append to, or store in, the datastore 301. Upon receipt of the request, the implementation 700 obtains704 an indicator of, e.g. pointer to, the current data store 301.Implementation 700 allows for multiple data stores 301 to be in use at agiven time, where each active data store is identified by an indicator,e.g. a pointer, which identifies at least the starting location wherethe identified data store 301 is located, e.g. in a memory 204. Once thelocation of the data store 301 is known, the implementation 700 returnsthe current value of an allocation counter (e.g., a starting offset)associated with the data store 301 to the requester and also,atomically, increments 706 the allocation counter by the amount of datarequested stored by the requestor. In other words, the value for theallocation counter returned to the requestor is the value of theallocation counter before the allocation counter is incremented with theamount of data requested stored by the requestor. The implementation 700then determines 708 whether the starting offset (e.g., before theallocation counter was incremented by the amount of data requestedstored) plus the amount of data requested stored is greater than themaximum file size of the data store 301.

If the starting offset plus the amount of data requested stored is lessthan the maximum file size of the data store 301, the implementation 700allows 710 the requesting thread/process to write its data in theallocated space of the data store 301. Once the requestingthread/process has completed its storage of the data, it willacknowledge to the implementation 700 that it has completed writing,upon receipt of which, the implementation 700 will account 711 for thecumulative number of bytes of data actually written to the data store301, e.g. by accumulating or otherwise incrementing the acknowledgmentcounter by the requested number of bytes.

If the starting offset plus the amount of data requested stored isgreater than the maximum file size of the data store 301, theimplementation 700 then determines 712 whether the starting offset(e.g., before the allocation counter was incremented by the amount ofdata requested stored) is less than or equal to the maximum file size ofthe data store 301.

If the starting offset is greater than the maximum file size of the datastore 301, the implementation 700 attempts 715 to atomically roll/swapthe currently active data store 301. Step 715 of implementation 700 mayrely upon a CAS-type operation to swap the active data store, namely, todetermine whether the active data store identifier that a requestor hasis the correct identifier for the active file (e.g., to ensure no otherprocess/thread has swapped the data store between the time the givenrequestor obtained the active data store identifier at step 704 and thetime that the given requestor attempts to swap the data store at step715).

If a given requestor's active data store identifier (obtained in step704) matches the active data store identifier obtained in step 715, thatgiven requestor is then allowed to process to step 716.

If a given requestor's active data store identifier (obtained in step704) does not match the active data store identifier obtained in step715, that given requestor may return to step 704 (e.g., a CAS failure)and attempt to write the data again, as shown in FIG. 7.

If the starting offset is less than or equal to the maximum file size ofthe data store 301, the implementation 700 determines that therequesting thread is the thread that should be responsible for cleaningup of the data store 301, and stores 714 the starting offset as a totalbytes allocated. Because the implementation has previously determined(at 708) that the number of bytes of data requested to be written willnot fit in the current data store 301, a new data store 301 must becreated or otherwise provided, referred to as a “roll” or “rolling thedata store”.

The implementation 700 then attempts 715 to atomically roll thecurrently active data store 301. Step 715 is then processed as discussedabove. As can be seen, two different requestors may arrive at step 715in two different processing paths: 712 to 715, and 714 to 715. Thus,step 715 runs the CAS type operation discussed above. Unlikeimplementation 700, implementation 600 does not include two differentways for two different requestors to arrive at a data swapping step.

The implementation 700 then determines 716 if the requestor should cleanup the data store 301, or if a separate (e.g., some other) thread shouldclean up the data store 301.

If the requestor should clean up the data store 301, the implementation700 then 718 waits until the acknowledgment counter associated with thedata store 301 reaches the stored total bytes allocated, to ensure thatall the allocated data is actually written to the data store 301. Inother words, the implementation 700 checks the cumulative number ofbytes actually written (e.g., indicated by the acknowledgment counter)to the current data store 301 against the stored total bytes allocatedto determine whether any other threads/processes, previously approved tostore data, are still in the process of storing their data in thecurrent data store 301. If the cumulative number of bytes written doesnot equal the number of allocated bytes, then another thread/processmust still be storing its data and the implementation 700 waits until itis determined that all other process/threads have completed storingtheir data to the current data store 301, i.e. that the cumulativenumber of bytes written (e.g., acknowledgment counter) equals the numberof bytes allocated (e.g., the stored total bytes allocated). After theacknowledgment counter associated with the data store 301 reaches thestored total bytes allocated, the data store 301 is closed 720 and theimplementation 700 returns to the input stage.

If a separate thread, e.g., a dedicated cleaning thread, should clean upthe data store 301, the separate thread may also perform checks (notshown) similar to the checks in 718 before cleaning the file. It shouldbe appreciated that it may be advantageous to delegate cleaning a datastore to a dedicated cleaning process, so that the thread that writesdata associated with an application (e.g., requestor 1) is not sloweddown by the task of cleaning a data store.

The illustrations of the embodiments described herein are intended toprovide a general understanding of the structure of the variousembodiments. The illustrations are not intended to serve as a completedescription of all of the elements and features of apparatus and systemsthat utilize the structures or methods described herein. Many otherembodiments may be apparent to those of skill in the art upon reviewingthe disclosure. Other embodiments may be utilized and derived from thedisclosure, such that structural and logical substitutions and changesmay be made without departing from the scope of the disclosure.Additionally, the illustrations are merely representational and may notbe drawn to scale. Certain proportions within the illustrations may beexaggerated, while other proportions may be minimized. Accordingly, thedisclosure and the figures are to be regarded as illustrative ratherthan restrictive.

While this specification contains many specifics, these should not beconstrued as limitations on the scope of the invention or of what may beclaimed, but rather as descriptions of features specific to particularembodiments of the invention. Certain features that are described inthis specification in the context of separate embodiments can also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable sub-combination. Moreover, although features may be describedas acting in certain combinations and even initially claimed as such,one or more features from a claimed combination can in some cases beexcised from the combination, and the claimed combination may bedirected to a sub-combination or variation of a sub-combination.

Similarly, while operations are depicted in the drawings and describedherein in a particular order, this should not be understood as requiringthat such operations be performed in the particular order shown or insequential order, or that all illustrated operations be performed, toachieve desirable results. In certain circumstances, multitasking andparallel processing may be advantageous. Moreover, the separation ofvarious system components in the described embodiments should not beunderstood as requiring such separation in all embodiments, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

One or more embodiments of the disclosure may be referred to herein,individually and/or collectively, by the term “invention” merely forconvenience and without intending to voluntarily limit the scope of thisapplication to any particular invention or inventive concept. Moreover,although specific embodiments have been illustrated and describedherein, it should be appreciated that any subsequent arrangementdesigned to achieve the same or similar purpose may be substituted forthe specific embodiments shown. This disclosure is intended to cover anyand all subsequent adaptations or variations of various embodiments.Combinations of the above embodiments, and other embodiments notspecifically described herein, will be apparent to those of skill in theart upon reviewing the description.

The Abstract of the Disclosure is provided to comply with 37 C.F.R. §1.72(b) and is submitted with the understanding that it will not be usedto interpret or limit the scope or meaning of the claims. In addition,in the foregoing Detailed Description, various features may be groupedtogether or described in a single embodiment for the purpose ofstreamlining the disclosure. This disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter may be directed toless than all of the features of any of the disclosed embodiments. Thus,the following claims are incorporated into the Detailed Description,with each claim standing on its own as defining separately claimedsubject matter.

It is therefore intended that the foregoing detailed description beregarded as illustrative rather than limiting, and that it be understoodthat it is the following claims, including all equivalents, that areintended to define the spirit and scope of this invention.

1. A computer implemented method of controlling storage of data in oneor more data stores, the method comprising: upon receiving one or morerequests from one or more requestors to store an amount of data:returning, to each requestor of the one or more requestors, a value ofan allocation counter associated with an active data store, the activedata store having a size and associated with an acknowledgment counter;and updating the value of the allocation counter based on each amount ofdata to be stored; determining, by each requestor, whether the returnedvalue of the allocation counter does not exceed the size of the activedata store and whether the returned value of the allocation counter plusthe amount of data requested stored by the requestor exceeds the size ofthe active data store; and upon determining, by any one requestor of theone or more requestors, that the returned value of the allocationcounter does not exceed the size of the active data store and that thereturned value of the allocation counter plus the amount of datarequested stored by the one requestor exceeds the size of the activedata store, designating, by the one requestor, a new data store as theactive data store.
 2. The computer implemented method of claim 1,further comprising causing, by the one requestor, the data storepreviously designated as the active data store to be cleaned.
 3. Thecomputer implemented method of claim 2, wherein the requestors otherthan the one requestor of the one or more requestors do not designate anew data store as the active data store or cause the data storepreviously designated as the active data store to be cleaned.
 4. Thecomputer implemented method of claim 2, further comprising waiting, bythe one requestor, until all the data from all the requests to storeamounts of data received before the one requestor's request to storedata has been stored in the data store previously designated as theactive data store before causing the data store previously designated asthe active data store to be cleaned.
 5. The computer implemented methodof claim 2, wherein the one requestor cleans the active data store. 6.The computer implemented method of claim 2, wherein the one requestorcauses a dedicated cleaning thread to clean the active data store. 7.The computer implemented method of claim 1, wherein the steps ofreturning the value of the allocation counter and updating the value ofthe allocation counter are performed atomically.
 8. The computerimplemented method of claim 7, wherein the step of returning the valueof the allocation counter is performed before the step of updating thevalue of the allocation counter.
 9. The computer implemented method ofclaim 1, wherein the allocation counter is indicative of portions of theactive data store that have been allocated for data storage in responseto requests to store amounts of data, and wherein the acknowledgmentcounter is indicative of portions of the active data store storing data.10. The computer implemented method of claim 9, further comprisingupdating a value of the acknowledgment counter upon storing data in theactive data store.
 11. The computer implemented method of claim 1,further comprising, upon determining by another requestor of the one ormore requestors that the returned value of the allocation counter plusthe amount of data requested stored by the another requestor does notexceed the size of the active data store, commencing storing the datarequested stored by the another requestor in the active data store. 12.The computer implemented method of claim 1, further comprising, upondetermining by another requestor of the one or more requestors that thereturned value of the allocation counter associated with the active datastore exceeds the size of the active data store, re-submitting, by theanother requestor, the another requestor's previously submitted requestto store data.
 13. The computer implemented method of claim 12, furthercomprising: receiving, by the another requestor, a returned value of anallocation counter associated with a new data store; and upondetermining by the another requestor that the returned value of theallocation counter associated with the new data store plus the amount ofdata requested stored by the another requestor does not exceed the sizeof the new data store, commencing storing the data requested stored bythe another requestor in the new data store.
 14. The computerimplemented method of claim 1, wherein the returning and updating stepsare performed via a LOCK XADD instruction.
 15. The computer implementedmethod of claim 1, wherein the data in the one or more requests to storedata is data received from a monitoring system of a data transactionprocessing system.
 16. A non-transitory computer-readable medium storinginstructions that, when executed by a processor, cause the processor to:transmit a request to store an amount of data; receive a value of anallocation counter associated with an active data store, the active datastore having a size; determine whether the received value of theallocation counter does not exceed the size of the active data store andwhether the received value of the allocation counter plus the amount ofdata requested stored exceeds the size of the active data store; andupon determining that the received value of the allocation counter doesnot exceed the size of the active data store and that the received valueof the allocation counter plus the amount of data requested storedexceeds the size of the active data store, designate a new data store asthe active data store.
 17. The non-transitory computer-readable mediumof claim 16, wherein the instructions further cause the processor tocause the data store previously designated as the active data store tobe cleaned.
 18. The non-transitory computer-readable medium of claim 17,wherein the instructions further cause the processor to wait until datafrom other prior requests to store amounts of data has been stored inthe data store previously designated as the active data store beforecausing the data store previously designated as the active data store tobe cleaned.
 19. The non-transitory computer-readable medium of claim 17,wherein the instructions further cause the processor to clean the activedata store.
 20. The non-transitory computer-readable medium of claim 17,wherein the instructions further cause the processor to cause adedicated cleaning processor to clean the active data store.
 21. Thenon-transitory computer-readable medium of claim 16, wherein theallocation counter is indicative of portions of the active data storethat have been allocated for data storage in response to requests tostore amounts of data.
 22. The non-transitory computer-readable mediumof claim 16, wherein the instructions further cause the processor to,upon determining that the received value of the allocation counterassociated with the active data store plus the amount of data requestedstored does not exceed the size of the active data store, commencestoring the data requested stored in the active data store.
 23. Thenon-transitory computer-readable medium of claim 16, wherein theinstructions further cause the processor to, upon determining that thereceived value of the allocation counter associated with the active datastore exceeds the size of the active data store, re-submit thepreviously submitted request to store data.
 24. The non-transitorycomputer-readable medium of claim 23, wherein the instructions furthercause the processor to: receive a value of an allocation counterassociated with a new data store; and upon determining that the receivedvalue of the allocation counter associated with the new data store plusthe amount of data requested stored does not exceed the size of the newdata store, commence storing the data requested stored in the new datastore.
 25. A computer system for controlling storage of data in one ormore data stores, the computer system comprising: means for receivingone or more requests from one or more requestors to store an amount ofdata; means for returning a value of an allocation counter associatedwith an active data store to each requestor of the one or morerequestors, the active data store having a size and associated with anacknowledgment counter; means for updating the value of the allocationcounter based on each amount of data to be stored; means fordetermining, by each requestor, whether the returned value of theallocation counter does not exceed the size of the active data store andwhether the returned value of the allocation counter plus the amount ofdata requested stored by the requestor exceeds the size of the activedata store; and upon determining, by any one requestor of the one ormore requestors, that the returned value of the allocation counter doesnot exceed the size of the active data store and that the returned valueof the allocation counter plus the amount of data requested stored bythe one requestor exceeds the size of the active data store, means fordesignating, by the one requestor, a new data store as the active datastore.